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This  report  describes  and  discusses  the  results  of  a 
5-year  program  evaluating  the  effectiveness  of  computer-assisted 
instruction  (CAI)  for  compensatory  education  of  elementary  school 
students.  The  report  comprises  sbven  parts:   (1)  an  overview  of  the 
final  report,   in  which  project  background,  methodology,  findings,  and 
implications  are  summarized;   (2)  a  descriptive  study  of  CAI  in 
compensatory  education,  in  which  the  setting  for  the  project,  the 
acquisition  and  operation  of  the  CAI  systems,  and  various 
perspectives  on  CAI  are  discussed;   (3)  a  description  of  the  CAI 
curricula  employed  during  the  project,   including  accounts  of  the 
mathematics,  reading,  and  language  arts  curricula,  student  time  on 
computers,  and  student  progress  in  CAI  curricula;   (4)  an  analysis  of 
the  effectiveness  of  CAI,  in  which  research  design  and  methodology, 
1-year  experimental  studies,'  longitudinal  studies,  and  treatment 
effects  are  discussed;   (5)  a  descriptive  review  of  longitudinal  ^ 
patterns  of  student  attitudes  toward  the  CAI  curriculum;  (6)  an 
assessment  of  CAI  as  an  instructional  medium  in  the  Los  Angeles 
County  Unified  School  District  setting;  and  (7)  an  evaluation  of  the 
costs  of  cAl  and  a  meta-cost-ef f ectiveness  analysis  of  educational 
intervention.  (JL) 
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The  project  has  produced  some  20  project  reports  over  its  years,  including 
yearly  reports  on  procedures  and  implementation  for  that  year  and  on  the 
year's  data  analysis.    However,  most  of  these  are  superceded  by  or  included 
in  the  seven-part  Final  Report  (bound  as  a  single  500-page  document).  Those 
most  relevant  include: 

#2.  Project  Handbook  for  CM  Coordinators,  December  1976  (by  Marjorie 

Ragosta  and  others). 
#11.  Assessing  Basic  Arithmetic  Skills  Across  Curricula,  December  1978 
(by  Donald  Alderman  and  others).    Also  in  The  Journal  of  Children's 
Mathematical  Behavior,  1979,  2(2). 
#14.  Computer-Assisted  Instruction  and  Compensatory  Education:  The  ETS/ 

LAUSD  Study.    An  Evaluation  of  the  Costs  of  Computer-Assisted  instruction, 
(by  Henry  M.  Levin  and  Louis  Woo),  October  1979.    Reprinted  as  Part  7A  of 
Final  Report  (Project  Report  #19),  and  also  in  Economics  of  Education 
Review,  1(1)  Winter  (1981),  1-25. 
#19.  Computer-Assisted  Instruction  and  Compensatory  Education:  The  ETS/ 
LAUSD  Study.    Final  Report,  April  1982  (by  Marjorie  Ragosta,  Paul  W. 
Dean  T.  Jamison). 
Overview  (36  pp.) 
Descriptive  Study  (122  pp.) 
The  CAI  Curriculums  (36  pp.) 

The  Effectiveness  of  CAI  (187  pp;  for  technical  reading) 
Longitudinal  Patterns  of  Student  Attitudes  in  a  Computer- 
Assisted  Instruction  Curriculum  (15  pp.)  (by  Philip  Griswold) 
Assessment  of  the  Effectiveness  of  Computer-Assisted 
Instruction  in  the  ETS -Los  Angeles  Strdy  and  a  Comparison 
of  CAI  with  Several  Intervention  Strategies  (65  pp;  for 
technical  reading)  (by  Gene  V.  Glass) 
An  Evaluation  of  the  Costs  of  Computer-Assisted  Instruction 
(34  pp)  (by  Henry  M  Levin  and  Louis  Woo) 
Part  7B.  Towards  a  Meta  Cost-Effectiveness  Analysis  of  Educational 

Interventions  (29  pp.)  (by  Henry  M.  Levin  and  William  Seidman) 
#20.  ComputeyAssisted  Instruction  and  Compensatory  Education:  The  ETS/LAUSD 
Study.    The  Executive  Summary  and  Policy  Implications,  June  1982  (by 
Marjorie  Ragosta,  Paul  W.  Holland,  and  Dean  T.  Jamison). 

The  project  also  served  as  the  basis  for  a  symposium  entitled  "Computer- 
Assisted  Instruction:  A  Longitudinal  Study"  at  the  Annual  Meetings  of 
Ithe  American  Educational  Research  Association,  April  1980,  chaired  by 
Robert  A  Davis  (U.  Illinois  at  Urbana.     Presenters 'included  Dean  Jamison 
(World  Bank),  Marjorie  Ragosta  (ETS),  Warren  Juhnke  (LAUSD  Dep.  Supt.), 
Roberta  Woodson  (LAUSD  Computer  Lab  Coordinator),  and  Paul  Holland  (ETS), 
with  a  critique  by  Henry  M.  Levin  (Stanford  U. ). 

The  project  director  also  has  a  presentation  scheduled  at  the  1983  Olontreal) 
Annual  Meetings  of  AERA  and  has  been  interviewed  for  several  projected 
articles  in  the  mass  media  (N.Y.  Times,  Electronic  Learning ,  ETS 
Developments,  and  the  premiere  issue  of  the  new  journal  Machine  Mediated 
Learning) . 
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Contents  of  the  Final  Report 


This  volume  is  the  final  report  of  a  five-year  study  of  computer- 
assisted  instruction  funded  by  the  National  Institute  of  Education  and 
conducted  by  Educational  Testing  Service  and  the  Los  Angeles  Unified 
School  District.     In  addition  to  the  Executive  Summary,  the  final  report 
ijs  composed  of  seven  parts,  the  contents  of  which  are  described  below: 
/         Part  1  Overview  of  the  Final  Report 

I         Part  2  A  Descriptive  Study 

Part  3  The  CAI  Curriculums 

Part  4  The  Effectiveness  of  CAI 

Part  5  Longitudinal  Patterns  of  Student  Attitudes  in  a 

Computer-Assisted  Instruction  Curriculum 
Part  6  Assessment  of  the  Effectiveness  of  Computer-Assisted 

Instruction  in  the  ETS-Los  Angeles  Study  and  a 
Comparison  of  CAI  with  Several  Intervention  Strategic 
Part  7        A.    An  Evaluation  of  the  Costs  of  Computer-Assisted 
=  Instruction 

B.    Towards  a  Meta  Cost-Effectiveness  Analysis  of 
Educational  Interventions 


Executive  Summary 

School  systems  concerned  with  compensatory  education  have  for  years 
sought  techniques  with  three  essential  characteristics:    effectiveness  > 
replicability ,  and  costs  within  typical  per-student  Title  I  allocations. 
Funded  by  the  National  Institute  of  Education,  Educational  Testing 
Service  (ETS)  in  conjunction  with  the  Los  Angeles  Unified  School  District 
(LAUSD)  spent  more  than  five  years  evaluating  computer-assisted  instruction 
(CAI)  for  compensatory  education.     The  results  were  heartening. 

Four  elementary  schools  were  equipped  with  CAI  labs  using  terminals 
and  printers  operated  by  a  minicomputer.     Computer-assisted  instruction 
was  provided  by  drill-and-practice  curriculums  in  mathematics,  reading 
and  language  arts  leased  from  Computer  Curriculum  Corporation  in  Palo 
Alto,  California.     Students  were  randomly  assigned  to  the  CAI  curriculums 
as  part  of  the  project's  research  design. 

The  curriculums  proved  to  be  effective  in  raising  students  scores  - 
not  only  on  tests  derived  from  the  CAI  curriculums  (CSTs)  but  on  stan- 
dardized tests  as  well.     Fourth  to  sixth-grade  students  who  received 
10  minutes  per  day  of  reading  CAI  performed  at  the  60th-65th  percentile 
of  students  without  reading  CAI  at  the  end  of  only  one  year.  Similarly, 
students  receiving  10  minutes  of  language-arts  CAI  performed  at  the 
54th-76th  percentile  of  students  without  language  arts.     In  most  cases 
those  performance  levels  were  maintained  over  three  years  with  continued 
use  of  CAI.     The  students  in  grades  1-6,  who  received  up  to  20  minutes 
per  day  of  mathematics  drill-and-practice  on  the  computer,  fared  even 
better.     On  standardized  tests  of  mathematics  computation^  CAI  students 
performed  at  the  64th  percentile  of  their  control  groups  at  the  end  of 
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only  one  year,  at  the  71st  percentile  by  the  end  of  two  years,  and  at  the 
76th  percentile  at  the  end  of  three  years*     On  the  CSTs  they  increased 
from  the  79th  percentile  in  year  1,  to  the  82nd  percentile  in  year  2, 
to  the  89th  percentile  by  the  end  of  year  3«     The  effectiveness  of  all 
three  CAI  curriculums  was  demonstrated* 

Replicabilir.y  was  demonstrated  by  fairly  consistent  results  over 
the  four  years  of  the  study*     Although  school  environment  and  personnel 
affect  the  CAI  program,  the  drill-and-practice  curriculums  themselves 
interacted  consistently  with  students  across  schools  and  across  years* 
The  major  inconsistency  was  the  suitability  of  the  reading  and  language 
arts  curriculums  for  all  elementary  school  students*     Some  students  in 
grades  4-6  were  unable  to  use  the  curriculums  because  they  could  not 
read  at  a  third  grade  level  and  a  few  were  so  fluent  in  English  that  they 
completed  the  courses  before  the  end  of  grade  6*    The  mathematics  curricu- 
lum, on  the  other  hand,  was  used  by  students  from  kindergarten  to  sixth 
grade  and  only  two  girls  completed  the  program  after  having  20  minutes 
per  day  of  instruction  from  grade  2  -  grade  5* 

The  per-pupil  cost  of  the  CAI  program  was  within  Title  I  allocations 
for  1977-78*     It  cost  about  $100,000  a  year  to  provide  a  classroom, 
personnel  and  equipment  for  operating  a  CAI  laboratory*     Slightly  more 
than  one^third  of  the  cost  was  for  facilities  and  equipment,  an  equivalent 
amount  was  spent  on  personnel  and  the  remainder  was  spent  on  curriculum 
rental,  maintenance  contracts  and  supplies*     One  10  minute  session  of 
CAI  daily  over  the  school  year  was  estimated  to  cost  about  $130  using  a 
CCC-17  minicomputer  to  operate  32  terminals*    Up  to  three  10  minute 


sessions  of  drill-and-practice  pould  be  provided  daily  for  each  disad- 
vantaged child  at  the  1911-1^  level  of  Title  I  expenditure. 

The  cost-effectiveness  of  CAI  vs.  other  Intervention  strategies 
was  unable  to  be  estimated  within  the  constraints  of  the  project.  The 
effectiveness  of  CM  was  compared  to  the  effectiveness  of  other  interven- 
tions:    reduction  in  class  size,  tutoring,  instructional  television  and 
electronic  calculators.    Although  the  effectiveness  of  the  mathematics  CAI" 
curriculum  appeared  to  approximate  the  effectiveness  of  tutorin'g,  the 
costs  of  interventions  other  than  CAI  were  not  immediately  available  for 
comparison. 

In  addition  to  effectiveness,  replicability  and  cost,  educational 

administrators  may  be  concerned  about  the  acceptance  of  computer-assisted 

s 

instruction  by  school  personnel.    Although  initial  acceptance'  by  teachers 
was  less  than  wholehearted,  by  the  end  of  the  study  most  tekchers  were 
convinced  of  the  value  of  CAI  and  supported  it  fully.    TKe  CAI  coordina- 
tors who  managed  the  CAI  labs  were  most  enthusiastic  about  the  help  that 
CAI  gave  in  improving  students'  skills.     Principals/enjoyed  bringing 
vislto-ps— the  CAI  labs  and  parents  filled  the  labs  at  every  opportunity,^ 
Students  enjoyed  the  CAI  program  as  well,  although  they  sometimes  complained 
of  the  restrictions  caused  by  the  research  design. 

CAI  is  real  boring,  but  I  guess  it  would 

be  better  if  I  had  reading  or  language, 

but  I'm  stuck  with  math. 
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Only  10-13  percent  of  students  had  any  negative  couiment,  however.  Most 

were  strongly  supportive: 

Computers  is  an  exciting  event,  every- 
body is  working  and  trying  hard  to  get  . 
one  hundred  percent*     It  feels  like 
we're  a  great  big  family,  just  doing 
our  jobs»  so  I  like  computers. 

I  think  the  CAI  program  is  fantastic. 

I  think  the  program  should  be  spread 

through  out  every  school  system  in 
America. 

The  overall  acceptance  of  the  CAI  program  in  Los  Angeles  was  excellent. 

In  1982,  two  years  after  the  government  support  for  the  project  ended, 

f. 

the  Los  Angeles  Unified  School  District  is  supporting  the  continued 
operation  of  the  CAI  labs. 
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INTRODUCTION 

With  the  passage  of  the  Elementary  and  Secondary  Education  Act  of 
1965,  the  Federal  Government  through  Title  I  assumed  a  major  role  in 
solving  the  nation's  problem  of  providing  compensatory  education  for 
disadvantaged  students.     The  search  began -for  a  technique  of  compensatory 
education  that  combined  three  essential  characteristics:  effectiveness 
in  substantially  improving  the  performance  in  basic  skills  of  the  compen- 
satory population,  replicability  beyond  the  original  site,  and  costs 
within  typical  per-student  Title  I  allocations, 

A  decade  of  developmental  work,  mostly  centered  at  Stanford  University' 
Institute  for  Mathematical  Studies,  resulted  in  a  set  of  three  computer- 
assisted  instruction  (CAI)  curriculums  for  elementary-school  compensatory 
education.    Early  studies  indicated  that  their  costs  fell  well  within 

per-pupil  Title  I  allocations^  and  that  use  of  the  mathematics  curriculum 

2 

over  a  period  of  one  year  improved  student  performance  in  mathematics. 
Since  the  curriculums  were  available  for  use  with  minicomputers  or  large 
moinframe  computers,  replicability  of  the  compensatory  intervention  could 
be  assured. 

If  these  CAI  curriculums  could  be  conclusively  shown,  over  a  period 
of  several  years,  to  provide  a  pedagogically  effective  intervention,  then 
state  and  local  educational  authorities  could  be  assured  of  having  at 
least  one  demonstrably  satisfactory  compensatory  intervention  at  their 
disposal.     If,  on  the  other  hand,  further  research  overturned  the  initially 
positive  findings,  unnecessary  outlays  on  this  form  of  CAI  could  be 
avoided. 
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A  joint  venture  funded  by  the  National  Institute  of  Education  (NIE) 
was  begun  in  1976  by  Educational  Testing  Service  (ETS)  and  the  Los 
Angeles  Unified  School  District  (LAUSD)  to  examine  these  issues.  The 
purpose  of  the  ETS/LAUSD  computer-assisted  instruction  study  was  to 
answer  definitely  the  question  of  whether  these  available  CAI  curriculums 
were  effective.     In  partilt^ular »  the  study  which  extended  over  more  than 
five  years »  was  desigited  to  answer  the  following  questions: 


(1)  Are  these  CAI  curriculums  effective  with  use  during 
one  school  year? 

(2)  Can  the  CAI  curriculums  continue  to  assist  students 
over  two  or  three  years  of  their  elementary-schooL, 
experience? 


Additional  questions  for  which  answers  were  sought  were  how  well  students 
and  school  personnel  accepted  the  CAI  program,  and  through  what  possible 
mechanisms  CAI  achieved  its  effectiveness.     In  order  to  strengthen  the 
findings  external  consultants  were  asked  to  do  a  cost-analysis  of  the  CAI 
program  as  used  in  the  study  and  to  evaluate  the  CAI  effectiveness  data 
frota  the  study  and  compare  its  effectiveness  with  other  intervention 
strategies. 

In  the  {executive  summary  of  the  results  presented  in  the  final  report, 
the  following  topics  will  be  covered: 
.     Background  and  Research  Design 
^  .     The  Effectiveness  of  the  CAI  Curriculums  . 
Costs  and  Cost-Effectiveness 
•     Perspectives  on  CAI  from  School  Personnel 
.     Implications  for  Schools 
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Background  and  Research  Design 

Four  elementary  schools  in  Area  D  (now  Area  4)  of  the  LAUSD  were 
selected  to  receive  CAI  labs,  and  two  additional  elementary  schools  were 
selected  to  provide  comparison  groups  of  students.    Work  on  the  construction 
of  the  CAI  labs  began  in  September,  1976,  and  the  labs  were  completed  in 
January,  1977.     For  the  remainder  of  that  school  year,  and  for  the  next  3 
years,  at  least  half  of  the  students  in  each  of  the  CAI  schools  received 
regular  drill-and-practicp  instruction. 

The  study  sought  the  most  widely  applicable  drill-and-practice 
computer  programs  in  basic  skills  available  for  elementary  schools  in 

1976.    The  following  drill-and-practice  CAI  curriculums  were  leased  from 

3 

Computer  Curriculum  Corporation  in  Palo  Alto,  California: 

Mathematics  Strands:     Grades  1-6.     (Containing  number  concepts, 
Matnemaci  horizontal  addition,  horizontal  subtraction, 

vertical  subtraction ,  equations ,, measurement , 
horizontal  multiplication,  laws  of  arithmetic, 
vertical  multiplication,  division,  fractions, 
decimals,  and  negative  numbers.) 

Reading:     Grades  3-6.  (Containing  word  attack,  vocabulary,  literal 

comprehension  and  work-study  skills.; 

Laneuage  Arts:     Grades  3-6.     (Containing  principal  parts;  verb  usage; 
Language  Arts.  ^object-verb  agreement;  pronoun  usage;  contractions, 

possessives  and  negatives;  modifiers  and  sentence 

structure. ) 

Reading-fbr  Comprehension  used  in  Grade  4,  year  4  only.     (Containing  the 
Reaaing  ro         p  ^  strands  in  the  older  reading  curriculum  plus  a 

paragraph  strand.) 

.Each  of  the  curriculums  was  composed  of  multipli  topics  or  strands  avail- 
able across  several  grade  levels.    The  computer  program  adapted  its  delivery 
of  each  strand  of  the  CAI  curriculum  to  the  performance  level  of  each  student 
and  moved  the  student  along  at  the  individual's  bwn  rate  of  progress. 


Multiple-Choice  or  open-ended  questions  were  presented  one  at  a  time  on  the 
terminal  screen  and  students  typed  in  their  responses.     At  the  end  of  each 
10-minute  session,  the  students/  scores  were  computed  including:     the  number 
of  items  atteupted.  the  number  of  correct  items,  and  the  percent  of  items 


correct. 
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The  Research  Design 

The  research  design  determined  not  only  which  students  would—or  / 
would  not— receive  CAI  but  also  determined  which  CAI  curriculum—or  , 
combination  of  curriculums-the  student  would  be  exposed  to.     In  the  CAI 
schools  in  year  1  all  students  in  grades  2.  4  and  6  received  CAI;  students 
in  grades  1.  3  and  5  served  as  cohort  controls  with  no  CAI.     In  subsequent 
years  alternate  waves  of  students  continued  to  receive—or  not  receive— CAI. 
The  result  was  12  one-year  studies  of  CAI  (three  each  year  for  four  years) 
which  combined  to  create  tl>ree  longitudinal  studies  of  CAI:    grades  1-3, 
grades  2-5  and  grades  4-6.  . 

A  more  important  control  group  was  provided  by  the  random  assignment 
of  students  to  their  CAI  curriculums  (See  Table  1).     In  grade  4.  year  1. 
for  example,  students  were  randomly  assigned  within  classrooms  either  to 
two  sessions  of  mathematics  CAI  (MM)  or  one  session  of  reading  and  one  of 
language  (RL)  or  one  session  of  mathematics  and  one  session  which  alternated 
between  reading  and  language  (MRL).     The  RL  group  served  as  a  control  for 
the  MM  and  MRL  groups  when  the  effectiveness  of  the  mathematics  CAI 
curriculum  was  being  assessed  and.  conversely,  the  MM  group  served  as  a 
control  in  studies  of  the  reading  and  language  CAI  curriculums.     The  CAI 
assignments  for  the  longitudinal  studies  are  described  below. 
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Table  1 
CAT  Treatment  Over  4  Years 
M  -  Mathematics  CAl.  R  -  Reading  CAI.  L  -  Language  CAI .  C  =  Reading  for  Comprehension  CAI) 


Grades  1-3  Only  the  mathematics  CAI  curriculum  was  available 

in  grades  1  and  2.     Classrooms  were  randomly  assigned 
to  the  CAI  labs  and  students  had  10  minutes  of  drill 
and  practice  in  mathematics  daily.     In  grade  3,  students 
were  randomly  assigned  within  classrooms  to  either 
mathematics  or  language  CAI  for  10  minutes  daily.  In 
this  longitudinal  study  students  were  randomly  assigned 
to  CAI  independently  each  year.    Assignments  were  not 
automatically  carried  over  from  one  year  to  the  next. 

Grades  2-5  In  grade  2  students  received  one  or  two  seven-minute  • 
sessions  of  mathematics  CAI.     In  grade  3  they  received 
two  lO-minute  sessions  of  CAI  daily ,  either  two  sessions 
of  mathematics  (MM)  or  one^  session  of  mathematics  and 
one  of  language  (ML).     Those  students  who  received 
ML  in  grade  3  received  reading  and  language  (RL)  during 
grades  4  and  5»  while  MM  students  continued  to  receive 
double  sessions  of  mathematics.    Once  students  were 
assigned  to  the  MM  curriculum  in  grade  3 »  they  continued 
their  assignment  through  grade  5  while  ML  students  in 
grade  3  converted  to  an  RL  assignment  for  grades  4  and  5. 

Grades  4-6  All  three  CAI  curriculums  were  available  to  this  group 
from  their  initiation  into  CAI.    One-third  of  the  group 
received  two  sessions  of  mathematics  CAI  daily  (MM) » 
one-third'  received  one  session  of  reading  and  one  of 
language  daily  (RL),  and  one-third  received  one  session 
of  mathematics  and  one  session  of  reading  alternating 
with  language  (MR/L).     Students  continued  their  fdurth- 
grade  assignments  in  grades  5  and  6»  while  new  students 
continued  to  be  randomly  assigned. 

The  two  one-year  'studies  not  covered  by  a  description  of  the  longitudinal 

studies  were  grade  6  in  year  1  and  grade  4  in  year  4.    The  latter  study 

was  designed  to  test  the  independent  effectiveness  of  the  reading  and 

language  curriculums  and  a  newer  read ing-for-comprehens ion  curriculum. 

Statistical  Methodology^ 

Students  were  tested  each  fall  and  spring  with  both  standardized 
tests — the  Iowa  Tests  of  Basic  Skills  (ITBS)  anjl  the  Comprehensive  Tests 
Basic  Skills  (CTBS) — and  curriculum  specific  tests  (CSTs).     The  CSTs  for 
each  grade  level  were  composed  of  100-120  questions  taken  directly  from 


each  of  the  CAI  curriculums.     The  roles  of  the  standardized  tests  and 


curriculum-specific  tests  were  different'.    The  CSTs  were  designed  specifi- 
cally to  measure  whether  or  not  each  o^  the  CAI  curriculums  was  successful 
in  improving  students'  performance  on  the  curricular  material.  Although 
the  CTBS  items  were  not  directly  related  to  the  CAI  curriculums,  the  CTBS 
was  used  as  a  standardized  measure  of  general  performance  in  mathematics, 
reading,  and  language  arts.     Treatment  effects  on  the  CSTs  estimate 
the  upper  bound  of  CAI  effectiveness  while  the  CTBS  effects  estimate  the 
lower  bound* 

'  The  summary  statistical  measure  of  the  effectiveness  of  the  various 
CAI  curriculums  was'  an  estimated  treatment  effect  derived  from  a  re- 
gression analysis  in  which  the  Effect  was  adjusted  for  pretest  scores, 
sex,  ethnicity  and  classroom  differences.     The  treatment  effect  was 
standardized  by  defining  it  as  that  proportion  of  the  residual  standard 
deviation  accounted  for  by  the  greater  (+)  or  lesser  (-)  numbers  of 
correct  responses  given  by  CAI  students.    The  standardized  treatment  effect 
was  used  to  average  the  effectiveness  of  CAI  across  varying  numbers  of 
studies.     Interpretation  of ' the  standardized  treatment  effect  derives  from 
widely  known  norms  for  the  rate  of  achievement  growth  per  month  of  typical 
schooling.    At  most  elementary  school  grades,  for  most  measures  of  educational 
achievement,  the  difference  between  the  average  pupil  at  the  beginning  and 
end  of  the  same  grade  is  about  1.00  standard  deviation  units.    One  month 
spent  in  school  accounts  on  the  average  for  a  growth  of  0.10  standard 
deviation  units.    In  addition,  the  treatment  effect  in  standard  deviation 
units  can  be  used  to  determine  (by  area  under  the  normal  curve)  the  percentile 
level  of  performance  of  one  group  over  another.     Thus,  a  0.10  standardized 
treatment  corresponds  to  the  54th  percentile  of  the  control  group;  a  standard- 
ized treatment  effect  of  0.30  is  equivalent  to  the  62nd  percentile. 
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The  Effectiveness  of  the  CAI  Curriculums 

The  three  CAI  curriculums — Mathematics  Strands,  Reading,  and  Language 
Arts — will  be  discussed  separately* 

,1 

Mathematics  Strands 

The  mathematics  strands  curriculum  had  the  length  and  breadth  to 
handle  all  of  the  students  in  the  CAI  study  over  its  4-year  duration. 
All  students  were  able  to  access  the  mathematics  curriculum.  Even 
kindergarten  students,  who  were  not  in  the  study,  visited  the  CAI  lab  in 
one  of  the  schools  on  a  regular  basis.     Only  two  students  completed  the 
mathematics  CAI  curriculum;  both  were  girls  who  had  been  in  the  program 
for  4  years,  receiving  20  minutes  of  mathematics  CAI  daily. 

Students  at  all  grade  levels  benefitted  from  the  CAI  mathematics 
curriculum  and  there  were  iniicatioQS  that  long«-term  students  showed  contin- 
uing gains  over  time.     There  were  12  one-year  studies  of  the  CAI  mathematics 
curriculum,  six  two-year  studies,  and  three  three-year  studies.    An  over- 
view of  the  mathematics  studies  is  presented  in  Table  2,  with  single-year 
studies  indicated  by  the  striped  rectangles  and  multi-year  studies 
indicated  by  the  longer  diagonal  lines. 

Table  2 
The  Mathematics  Studies 


YEAR 

2 
3 

k 


1 

2 

GRADES 
3     1  h 

5 

6  J 

m 

> 

m. 

m 

F 

1 
1 
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Statistical  results.    There  were  four  measures  of  CAI  treatment  effects 

in  mathematics:     the  curriculum-specific  test  (CST),  and  three  subtests  of 

the  CTBS — computation,  concepts  and  applications.  "For  each  of  the  ntea-&ures, 

treatment  effects  for  mathematics  CAI  were  defined  in  terms  of  standard 

deviations  above  or^elow  the  adjusted  mean  performance  of  students  without 

mathematics  CAI  and  were  average3"for  one-year,  two-year,  and  three-year 

studies.     The  results  are  presented  in  Table  3. 

Table  3  , 

Summary  of  th^  1-Year,  2-Year,  and  3-Year 
Studies  of  Mathematics  CAI 

Number  of 

*  Studies  Standardized         Performance  Level 

Averaged  Treatment  Effects         in  Percentiles 


Tests 

lYR 

•  2YR 

3YR 

lYR 

2YR 

3YR 

lYR 

2y.R 

3YR 

Math  CST) 

12 

6 

3 

.80** 

.91** 

1.23** 

79 

82 

,89 

CTBS  Computation 

9 

6 

2 

.36** 

.56** 

.72**, 

64 

.71 

76 

CTBS  Concepts 

7 

5 

2 

-.02 

.12 

.09 

49 

55 

54 

CTBS  Applications 

7 

5 

2 

.03 

.12 

.26 

51 

55 

60 

CTBS  Concepts  & 

2 

1 

0 

.34** 

.50 

63 

69 

Applications 


**    p  <  .Ol. 

^In  grades  1  and  2  Concepts  and  Applications  is  a  single  subtest. 

For  the  CSTs,  the  average  treatment  effect  for  the  12  one-year  studies 
was  .80  of  a  standard  deviation,  indicating  that  students  receiving  mathematics 
CAI  were,  on  the  average,  four-fifths  of  a  standard  deviation  higher  than 
other  students  in  mathematics  performance  on  the  CST  at  the  end  of  one  year. 
For  the  6  two-year  studies,  the  average  treatment  effect  was  .91  and  for  the 
3  three-year  studies  was  1.23.     Over  time,  the  mathematics  CAI  groups 
increased  their  mean  distance  from  the  non-mathematics  CAI  groups  on  the 
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tests  of  the  CAl  curriculum*    The  CST  results  indicate  that  the  mathematics 
strands  curriculum  was" effective  in  giving  students  drill-and-practice  in 
mathematics  computation.     Students  in  the  mathematics  CAI  curriculum  outper- 
formed other  students  on  the  test  of  their  CAI  curriculum.     Results  for  the 
standardized  test  are  less  striking  but  similar. 

On  the  standardized  test — the  CTBS-^the  12  one-year  studies  showed  an 
average  treatment  effect  of  .31  standard  deviation  on  mathematics  computation 
That  figure  rose  to  an  average  of  .36  when  first  year  studies  were  omitted 
(because  testing  occurred  only  two  months  after  the  CAI  labs  opened).  For 
the  6  two-year  studies  the  average  treatment  effect  was  .56,  and  for  the  2 
three-year  studies  was  .11.  Ojer  time,  the  mathematics  CAI  groups  increased 
their  distance  from  the  non-mathematics  CAI  groups  on  the  CTBS  computation 
subtest. 

On  the  CTBS  concepts  and  applications  subtests,  the  results  were  less 
clear.     In  grades  1  and  2  concepts  and  application  problems  comprise  one 
subtest.    CAl  treatment  effects  in  the  3  one-year  studies  aVeraged  one-third 
of  a  standard  deviation  and  were  statistically  significant.     The  one  two-year 
study  reported  a  treatment  effect  of  .50.     Of  the  9  one-year  studies  at 
grade  3  or  higher,  the  mean  CAI  treatment  effects  on  mathematics  concepts 
and  applications  were  close  to  zero.     When  the  5  two-year  studies  were 
averaged^  the  mean  effects  for  concepts  and  applications  were  .12  and  .12. 
Mean  treatment  effects  for  concepts  and  applications  in  the  2  three-year 
studies  were  .09  and  .26. 

Discussion.     The  Mathematics  Strands  curriculum  was  designed  to  provide 
drill-and-practice  CAI  in  computation.     The  CST  and  the  CTBS  computation 
subtest  provided  the  best  indicators  of  the  curriculum's  effectiveness  in 
helping  students  improve  computational  skills.     Results  from  both  measures 
indicated  significant  treatment  effects  due  to  use  of    Mathematics  Strands 


curriculum  for  only  one  year  and  increasing  treatment  effects  with  its  use 

■  r 

over  tv/o  and  three  years.     The  CST  and  the  CTBS  computation  subtest  tell 
similar  stories;  they  differ  only  in  the  degree  to  which  they  separate 
mathematics  CAI  students  from  their  CAI  controls.     Plots  of  normal  curves 
derived  from  the  standardized  treatment  effects  for  the  CST  and  CTBS  computa- 
tion subtests  are  presented  in  Figures  1  and  2.     Both  plots  show  increasing 
separation  over  time  between  scores  of  users  and  non-users  of  the  mathematics 
CAI  curriculum.    The  CST  data  show  greater  gains  than  the  CTBS  data,  consistent 
with  earlier  statements  that  the  CSTs  might  represent  an  upper  bound  of  true 
treatment  effects. 

The  mean  performance ^of  mathematics-CAI  students  on  the  CSTs  was  at  the 
79th  percentile  of  control  students  at  the  end  of  one  year,  at  the  82nd 
percentile  at  the  end  of  two  years  and  at  the  89th  percentile  at  the  end  of 
three  years.    -Fpl*  the  CTBS  computation  data  mean  performance  levels  of 
mathematics  CAI  students  were  at  the  64th  percentile  of  control  students 
after  one  year,  7lst  percentile  after  two  years  and  76th  percentile  after 
three  years. 

The  treatment  effects  for  students  in  grades  3-6  on  Concepts  and 
Applications  may  be  underestimates  of  the  true  effect  in  mathematics 
since  the  subtests  were  composed  of  word  problems  which  required  both 
mathematical  and  reading  ability  and  the  (within  CAI)  control  groups  in 
these  analyses  had  received  reading  and  language  arts  CAI.     A  separate 
but  coordinated  investigation  into  the  solution  processes  of  users  and 
non-users  9f  the  mathematics  CA,I  curriculum^  found  no  differences  between 
the  two  groups  in  their  understanding  of  mathematics.     However,  the  data 
from  two-year  and  three-year  studies  presented  here  would  seem  to  indicate 
that  the  CAI  mathematics  curriculum  was  helpful  in  increasing  scores  on  the 
applications  subtest  and  perhaps  on  the  Concepts  subtest  as  well. 
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Figures  1  and  2 


EFFECTS  OF  HATH  Cf^I  ON  MATH  CST»S  OUER  3  YRS 
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Figure  1 


EFFECTS  OF  WATH  CAI  OH  CTBS  COMPUTATION  OUER  3  YRS 
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An  additional  pattern  emerged  from  the  longitudinal  data  for  grades 
4-6  where  some  students  received  two  sessions  of  mathematics  CAI  (the  MM 
group)  while  others  received  only  one  session  (the  MRL  group).  Treatment 
effects  for  the  MRL  group  were  about  half  the  size  of  the  treatment 
effects  for  the  MM  gro^up  both  on  the  mathematics  CSTs  and  the  computation 
subtest  of  the  CTBS.  ' 

Overall,  the  mathematics  strands  curriculum  performed  very  well.  It 
adapted  to  students  of  all  ability  levels  and  provided  effective  drill- 
and-practice  in  mathematics  computation.     Tlie  effectiveness  of  the 
curriculum  was  demonstrated  both  in  one-year  studies  and  over  two-  and 
three-year  periods.     Increased  amounts  of  mathematics  CAI  either  within 
the  school  year  or  across  school  years  were  associated  with  higher  test 
scores  both  on  curriculum-specific  tests  and  standardized  tests. 

Reading  and  Language  Arts 

The  CAI  reading  curriculum  used  in  thie  study  was  developed  for 
students  in  grades  3-6  and  contained  a  strand  of  basic  sentences  which 
purported  to  reach  students  at  grade  level  2.5.    For  the  students  in  our 
study,  those  estimates  were  misleading.    At  grade  4  in  year  1,  many 
students  assigned  to  the  reading-language  (RL)  CAI  treatment  were  trans- 
ferred to  the  MM  group  because  they  were  non-English-speaking,  non-readers, 
or  limited-English-speaking  students  who  did  not  read  English.  Those 
students  were  then  excluded  from  the  analyses.     It  was  clearly  a  drawback 
that  accessibility  to  the  reading  drill-and-practice  CAI  curriculum  was 
dependent  on  an  ability  to  read  relatively  well.    At  the  other  extreme, 
in  grade  6  the  reading  CAI  curriculum  sometimes  proved  to  be  too  easy.  A 
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few  students  "topped  out"  of  the  reading  curriculum  (i*e.,  completed  it) 
during  the  early  rapid-motion  phase  in  which  the  computer  determines  a 
student^s  initial  ability  level.     A  few  students  assigned  to  2  sessions 
cf  reading  daily  topped  out  within  a  few  months  although,  generally 
speaking,  progress  was  very  slow  in  the  reading  curriculum. 

The  language  curriculum  was  designed  for  students  in  grades  3-6  and 
because  its  vocabulary  was  simpler  than  that  in  the  reading  curriculum  it 
caused  fewer  problems  of  accessibility  in  the  early  grades.     On  the  other 
hand,  it  caused  more  frequent  problems  in  grade  6  T?ith  students  topping 
out. ^  Relatively  large  numbers  of  students  who  were  assigned  to  2 
sessions  of  language  CAI  daily  in  the  latter  half  of  sixth  grade  topped 
out  of  the  curriculum  in  the  rapid  motion  phase  and  others  topped  out 
within  a  few  months.    Progress  in  the  language  curriculum  was  more  rapid 
than  progress  in  reading.     For  students  interested  in  achieving,  the  more 
rapid  progress  was  pleasing.     Of  the  students  assigned  to  both  reading 
and  language  arts,  most  preferred  the  language  curriculum  because  of  the 
more  rapid  movement. 

The  length  and  breadth  of  the  reading  and  language  CAI  curriculums 
did  not  purport  to  be  as  great  as  the  mathematics  CAI  curriculum  and 
perhaps  this  is  less  of  a  problem  for  the  use  of  the  curriculums  in 
elementary  schools  than  it  was  for  the  evaluation  of  the  curriculums  in 
this  study.     Schools,  after  all,  do  not  generally  assign  students  randomly 
to  a  CAI  curriculum.    However,  there  were — and  are — limitations  to  the 
use  of  these  specific  reading  and  language  curriculums  even  though  they 
were  the  broadest  available  when  the  study  started.     Perhaps  with  newer 
technology  a  reading/language  CAI  curriculum  could  be  built  which  would 


have  the  broad  applicability  (grades  1-6)  that  the  mathematics  strands 
curriculum  enjoys. 

Reading.    Table  4  summarizes  the  one-,  two-,  and  three-year  studies 
of  reading.     Since  the  CAI  reading  curriculum  was  used  only  in  grades 
4-6,  there  were  only  seven  one-year  studies,  three  two-year  studies,  and 
one  three-year  study • 


On  curriculum-specific  tests  of  reading,  the  mean  treatment  effect 
for  the  seven  one-year  studies  was  .44  indicating  that  students  exposed 
to  reading  CAI  for  one  year  averaged  more  than  two-fifths  of  a  standard 
deviation  higher  than  students  exposed  to  other  CAI  curriculums.  The 
pattern  of  the  seven  scores,  however,  may  be  more  enlightening  than  the 
mean.     The  treatment  effects  for  grade  4  in  all  three  years  and  for  grade 
6  in  year  1  were  all  statistically  significant  and  averaged  .59.  The 
treatment  effects  for  grade  5  in  both  years  and  grade  6  in  year  3  were 
not  significant  and  averaged  .29.     The  significant  effects  were  obtained 
when  students  were  exposed  to  reading  CAI  for  the  first  time.     In  years 


subsequent  to  the  first  year  of  exposure,  the  reading  CAI  students 
continued  to  perform  better  than  mathematics  CAI  students  both  on  the 
pretest  and  posttest  administrations  of  the  reading  CST  but  they  did  not 
increase  their  gains  significantly.    The  three  two-year  studies  showed  a 
mean  treatment  effect  of  .52  while  the  single  three-year  study  had  a  ' 
treatment  effect  of  .42.     Summaries  of  the  mean  CAI  treatment  effects  for 
students  receiving  reading  CAI  are  given  in  Table  5. 

Table  5  - 

Summary  of  the  1-Year,  2-Year,  and  3-Year 
Studies  of  Reading  CAI 


Tests 

Reading  CST 
CTBS  Vocabulary 
CTBS  Comprehension 


Number  of 

Studies 

Averaged 

lYR  2YR  3YR 

8  3  1 

8  3  1 

8  3  1 


Mean  Standardized 
Treatment  Effects 

lYR  2YR  3YR 

.38**  .52**  .42 

.25**  .17  .58 

.23**  -.01  /'-.2A 


Performance  Level 
in  Percentiles 

lYR  2YR  3YR 

65  70  66 

60  57  72 

59  50  41 


P  <  .01. 


On  the  vocabulary  and  comprehension,  subtests  of  the  CTBS,  results 
were  mixed.     On  the  basis  of  the  single  three-year  study,  one  might  reach 
the  conclusion  that  the  vocabulary  skills  of  CAI  students  improved  over 
three  years.     But  the  data  also  seem  to  indicate  decreasing  ability  of 
CAI  students  to  deal  with  reading  comprehension.    Whether  a  consistent 
pattern  failed  to  emerge  because  of  some  quality  of  the  curriculum,  the 
effects  of  the  bottoming-out  and  topping-out  phenomena  on  the  research 
design,  or  some  other  factor,  is  not  immediately  obvious. 


During  the  final  year  of  the  CAI  study  a  newer  reading-for-compre- 
hension  curriculum  was  evaluated  along  with  the  older  reading  curriculum. 
The  newer  curriculum  resembled  its  predecessor  except  that  a  strand  of  para- 
graph comprehension,  had  been  added.     Students  who  were  assigned  to  the 
reading-for-comprehension  curriculum  did  better  than  students  assigned  to 
reading,  language  arts  or  mathematics  CAI  both  on  the  reading  comprehension 
and  on  the  language  expression  subtests  of  the  CTBS. 

Language.  Table  6  summarizes  the  one,  two  and  three-year  studies  of 
language  arts  CAI.  There  were  9  one-year  studies,  4  two-year  studies  and 
2  three-year  studies. 

Table  6 
Language  Studies 

GRADES 


As  was  the  case  with  the  reading  results,  the  CST  treatment 
effects  in  the  first  year  of  the  study  were  the  largest  obtained  in  the 
four  years  even  though  students  had  received  only  four  months  of  CAI  in 
that  school  year.     However,  all  treatment  effects  in  one-year  studies 
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involving  the  language'  CSTs  were  statistically  significant  for  students 
receiving  10  minutes^ of  language  arts  CAI  daily.     There  was  a  greater 
differentiation  between  the  MM  and  .RL  groups  when  tested  on  the  content 
of  the  language  curriculum  than  the  reading  curriculum.    Mean  CAI  treatment 
effects  for  one-,  two-  and  three-year  studies  of  the  language  curriculum 
are  presented  in  Table  7. 

Table  .?  •  , 

Summary  of  the  1-Year,  2-Year,  and  3-Year 
Studies  of  Language  CAI 


Number  of 
Studies  ' 
Averaged 


Mean  Standardized 
Treatment  Effects 


Performance  Level 
in  Percentiles 


Tests 

lYR 

2YR 

3YR 

lYR 

2YR 

3YR 

lYR 

2YR 

3YR 

Language  CST 

10 

4 

2 

.71** 

.76** 

.73** 

76 

78 

77 

CTBS  Spelling 

10 

4 

2 

.14* 

.05 

.14 

56 

52 

56 

CTBS  Mechanics 

10 

4 

2 

.22** 

.27* 

.25 

59 

61 

60 

CTBS  Expression 

10 

4 

2 

.11 

.05 

.23 

54 

52 

59 

*  p  <  .05. 
**    p  <  .01. 


ERIC 


Mean  treatment  effects  were  averaged  over  10  sets  of  data  in  the  9 
one-year  studies  (RL  or  LL  treatments),  4  sets  of  two-year  studies  and 
two  sets  of  three-year  studies.     Although  all  the  treatment  effects  were 
positive,  they  failed  to  show  this  pattern  of  increasing  gains  over  three 
years  demonstrated  by  the  mathematics  CAI  curriculum.     This  may  be  due  in 
part  to  the  problems  encountered  in  the  random  assignment  of  students  to 
reading^nd  language  CAI  curriculums.     It  may  also  be  due  to  the  fact 
that  RL  students  in  grades'  4-6  received  only  half  as  much  time  in  reading 
CAI  or  language  CAI  as  the  students  received  in  mathematics  CAI.  For 


^2 


whatever  reason,  CAI  students  in  the  reading  and  language  arts  curriculuns 
failed  to  demonstrate  increasingly  better  test  performance  over  multiple 
years.    However,  each  of  the  verbally  oriented  curriculums  did  demonstrate 
an  ability  to  help  students  improve  reading  and  language  scores  as 
measured  by  curriculum-specific  tests  and  by  standardized  tests. 

In  summary,  each  of  the  CAI  curriculums  proved  its  effectiveness 
although  some  curriculums  performed  better  than  others.     The  mathematics 

strands  curriculum  showed  strong  ^^omise  in  longitudinal  studies.  The 

\> 

reading  and  language  CAI  curriculums^  had  less  breadth  but  were  both 
capable  of  helping  students  to  improve\^   Computer-assisted  instruction  as 
defined  and  used  in  this  study  was  a  pow^ful  tool  for  increasing  students 
skills  in  mathematics,  reading  and  language  arts. 

9 

CAI -Effectiveness :     A  Second  Look 

Gene  Glass  evaluated  the  CAI  study  and  its  measurements,  estimated  , 
the  effects  of  added  CAI  and  the  effects  of  replacing^  some  traditional 
instruction  by  CAI  and,  finally,  compared  the  effectiveness  of  CAI  with 
alternative' methods  of  improving  achievement. 

He  evaluated  the  usefulness,  of  the  various  control  groups  in  the  same 
way  as  the  primary  investigators— relying  most  heavily  on  the  within-CAI 
controls  derived  from  the  random  assignment  of  students  to/CAI  curriculums 
considering  the  data  from  the  cohort  control  studies  useful  despite  their 
drawbacks,  but  rejecting  the  studies  involving  comparison  schools.  He 
evaluated  the  standardized-test  data  superior  to  the  curriculum-specific 
test  (CST)  data  in  estimating  true  treatment  effects. 
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Glass  used  meta-analytic  techniques  to  estimate  CAl  treatment  effects 
on  the  standardized  tests.     He  analyzed  the  treatment  effects  from  the 
randomized  studies  as  effects  of  added  CAl  using  the  rationale  that  CAl 
students  had  equal  exposure  to  whatever  was  taught  in  the  classroom  but 

differential  exposure  to  CAl  curriculums  in  the  CAl  lab.     The  effect 

1 

sizes  for  10-20  minutes  of  reading  and  language-arts  CAl  for  one  year 
ranged  from  a  low  of  .15  of  a  standard  deviation  in  spelling  to  a  high  of 
•  45  in 'reading  comprehension.     For  one  year  of  mathematics  CAl,  treatment 
effects  were  estimated  at  1/4  of  a  st!andard  deviation  for  10  minutes  of 
CAl  daily  and  1/2  of  a  standard  deviation  for  20  minutes  daily. 

The  effects  of  replacing  part  of  traditional  instruction  by  CAl  were 
determined  by  a  meta-analysis  of  several  CAl  vs.  cohort-control  studies. 
The  treatment  effects  were,  in  general,  slightly  lower  for  the  cohort- 
control  studies  with  the  largest  effects  occurring  for  mathematics 
computation  (.45  of  a  standard  deviation)  and  language  mechanics  (1/4  of 
a  standard  deviation). 

Glass  compared  the  CAl  treatment  effects  with  effects  obtained  using 
other  intervention  strategies:     reduction  in  class  size,  tutoring, 
instructional  television,  teacher-training,  and  electronic  calculators. 
While  the  effectiveness  data  from  the  alternative  methods  of  instruction 
helped  to  place  the  CAl  data  in  a  slightly  more  informative  context,  no 
attempt  was- made  to  evaluate  costs.    Although  the  effectiveness  of  the 
mathematics  CAl  curriculum,  fot  example,  appeared  to  approximate  the 
effectiveness  of  mathematics  tutoring  (the  ultimate  reduction  in  class 
size),  the  instructional  method  of  choice  should  be  the  one  which  is  most 

cost  effective.    Closure  on  the  question  of  cost-ef f^ctiveness  must  await 

i 

further  analysis.  !  ' 
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CAI:     The  Cost  Study 

Given  the  study's  focus  on  the  educational  needs  of  disadvantaged 
students,  two  questions  arose  pertaining  to  costs.     The  first  question 
was  based  upon  the  assumption  that  funding  for  special  educational 
services  for  disadvantaged  students  is  derived  primarily  from  special 
categorical  aid  for  that  purpose,  such  as  that  received  under  Title  I  of' 
the  Elementary  and  Secondary  Education  Act  of  1965.     Therefore,  it  is 
important  to  know  if  CAI  can  be  provided  within  the  budget  that  is 
available  for  these  compensatory  educational  services  for  disadvantaged 
youngsters.     Second,  it  is  important  to  know  if  the  CAI  approach  can 
improve  the  educational  proficiencies  of  disadvantaged  students  at  costs 
that  are  similar  to  or  less  than  those  associated  with  other  instructional 
alternatives. 

The  first  issue  is  one  of  cost  feasibility.     If  the  costs  of  this 
CAI  approach  exceed  the  funds  available  for  instructional  purposes  for 
disadvantaged  youngsters,  it  will  not  be  within  the  boundaries  Of  feasi- 
bility.    The  second  issue  is  one  of  cost  e::f ectiveness.     Even  if  CAI  can 
be  provided  within  the  present  budgets  fov  compensatory  education,  it 
should  be  adopted  only  if  it  provides  better  results  relative  to  its 
costs  than  do  existing  alternatives. 

Cost  feasibility  was  examined  by  Levin  and  Woo  by  estimating  the 
replication  costs  of  the  CAI  approach  used  in  the  ETS/LAUSD  experiment, 
that  is,  the. cost  of  replicating  that  system  in  other  school  settings. 
Costs  were  limited  to  those  associated  with  the  delivery  of  CAI  while 
omitting  costs  that  were  tied  uniquely  to  the  experimental  status  of  the 
present  system. 

3  J 
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Based  upon  the  Ingredients  approach  to  cost  analysis,  it  was  found 
that  up  to  three  sessions  of  drill  and  practice  of  10  minutes  duration 
could  be  provided  daily  for  each  disadvantaged  child  at  the  1977-78 
level  of  Title  I  expenditures*    This  means  that  three  different  subjects 
could  be  provided,  or  that  multiple  sessions  in  one  or  two  subjects  could 
be  offered^  for  each  child.    As  such,  it  appears  that  the  instructional 
strategy  is  cost  feasibl^e  within  present  provisions  for  compensatory 
education.    Utilizing  one  A-16  minicomputer  to  operate  CAI  labs  in  two 
schools  increased  costs  rather  substantially,  but  two  sessions  of  CAI 
daily  would  still  be  feasible  within  1977-197,8  compensatory  educational 
allocations. 

Levin  and  Woo  also  estimated  costs  for  the  more  advanced  CCC-17 
minicomputer  system,  and  somewhat  surprisingly  the  costs  were  in  the  same 
range  as  those  of  the  older  A-16.    One  10-minute  session  of  CAI  daily 
over  the  school  year  was  estimated  to  cost  about  $130.     In  part,  this 
finding  reflects  the  very  heavy  software  component  of  CAI  approaches, 
and,  in  part,  it  may  reflect  the  possibility  that  the  CCC-17  is  more 
effective  than  the  A-16.  ' 

In  1977-78  it  cost  about  $100,000  a  year  to  provide  a  classroom, 

personnel  and  equipment  for  servicing  32  terminals  in  a  CAI  laboratory  using 

an  A-16  minicomputer.     A  reported  breakdown  of  cost  into  five  categories 

is,  accompanied  by  percentages  of  funds  expended  for  each  category; 

Facilities  and  equipment  (including  a  36% 
classroom,  computer,  32  terminals  and  printer) 

Personnel  (including  a  CAI  coordinator  and  35%  \ 

I  two  3-hour  teaching  assistants) 

Curriculum  rental  (from  CCC)  '  7% 

ERLC 


Maintenance  (contracts  for  computer,  16% 
terminals  and  printer) 

Miscellaneous  factors  (supplies,  6% 
Insurances,  etc*)  . 

More  than  one-third  of  the  cost  was  associated  with  personnel* 

It  Is  clear  that  a  more  exhaustive  analysis  of  the  merits  of  different 
CAI-^  approaches i  as  well  as  a  comparison  between  them  and  other  Instructional 
strategies,  will  require  effectiveness  data  as  well  as  cost  estimates. 
Given  the  data  comparing  the  effectiveness  of  the  current  CAI  approach  with 
other  intervention  strategies,  it  is  hoped  that  a  cost  effectiveness  comparison 
can  be  made  at  some  future  date.     The  methodology  for  the  cost-effective- 
ness study  is  Included  in  the  Final  Report. 

11  ^ 

Perspectives  on  CAI  from  School  Personnel 
« 

The  CAI  program  described  in  this  summary  was  well  received  by 
most  school  personnel.     Principals  liked  the  program  and  often  brought 
visitors  to  the  CAI  labs.    Parents  filled  the  lab  during  every  open  house 
and  were  enthusiastic  supporters  of  the  program.    Teachers,  students  and 
CAI  coordinators  were  most  closely  involved  with  the  program.  Their 
perspectives  will  be  discussed  separately. 

Teachers '  Perspectives 

Initial  acceptance  of  the  CAI  program  on  the  part  of  teachers 
was  less  than  wholehearted  due,  at  least  in  part,  to  start-up  problems 
with  the  equipment  and  interruptions  to  classroom  organization  by  beginning 
the  CAI  program  in  mid-year.     By  the  end  of  the  first  year  some  teachers 
were  enthusiastic  while  others  had  reservations.    By  the  time  the  study 
ended,  most  teachers  supported  CAI  fully  and  were  convinced  of  its 


benefits  to  students*    There  were  occasional  complaints  of  less-than-* 
perfect  performance  in  two  areas:    disruption  of  the  classroom  by  the  CAI 
schedule  and  dependability  of  the  equipment.     Even  in  these  areas,  ratings 
were  much  more  positive  than  negative.     Overall,  the  CAI  program  received 
very  high  ratings  for  being  well-run,  helpful,  worthwhile,  and  successful. 


Students 

Students  were  very  enthusiastic  about  computer^assisted  instruction. 

They  were  not  always  enthusiastic  about  the  specific  assignment  they 

received  during  the  random  assignment  of  students  to  curriculums,  especially 

when  that  assignment  continued  for  several  years.     Examples  of  the 

strongest  negative  responses  from  students  when  asked  about  the  CAI 

program  included  the  following: 

CAI  is  real  boring,  but  I  guess  it  would  be  better  if  I  . 
had  reading  or  language,  but  I'm  stuck  with  math. 

it  it  it  it  it 

I  dont  think  computers  are  very  fun  I  know  that  I'm  learning 
more  than  I  would  in  my  class  aloan  but  I  find  CAI  boring 
ybu  keep  ori  getting  question  after  question  I  take  only  Math-M 
and  would  find  CAI  more  interesting  if  If  I  could  do  some  thing 
a  little  different  once  and  a  while.     Truly,  K.H. 

Only  10-13  percent  of  student  comments  in  any  year  were  negative,  however. 

Most  students  were  strongly  supportive  of  the  program.     The  students 

speak  for  themselves  without  editorial  assistance: 

I  think  that  the  Computer  is  fun  and  it  can  help  you  learn, 
dhd  to  be  smarter  in  class.    A  computer  is  a  machine  that 
give  you  all  the  answers.  A  terminal  is  something  that  gets 
it  questions  from  the  computer  A  computer  is  a  nice  thing  to 
have.     It  helps  you  in  what  you  need  to  work  on.     And  a 
Computer  is  like  a  teacher.     It  asks  you  questions  and  you 
answer  it.     I  think  a  computer  is  more  smarter  than  you  are 
and  it  is  smarter  than  anything. 

***** 


I  think  the  computor  is  allmost  the  best  class  in  this  whole 
school.  For  one  reason  it  helps  you  to  understand  things 
that  you  dont  understand  and  it  helps  you  to  know  things 
that  you  don't  know.    You  want  to  know.     One  more  thing  is  I 
like  the  teacher.     They  are  very  nice  if  you  don't  know 
something  all  you  have  to  do  is  raise  your  hand  and  they 
will  help  you.     I  am  very  happy  that  we  have  computor  in  our 
school.     1  wish  that  we  would  have  got  it  earlier.  I'm  also 
^   glad  that  we  get  to  go.  everyday  of  the  week.  ^ 

*   A   A   A  * 

Computers  is  an  exiting  event,  everybody  is  working  and 
trying  hard  to  get  one  hundred  percent*     It  feels  like  we're 
a  great  big  family,  just  doing  our  jobs,  so  I  like  computers 
even  though  sometimes  I  get  a  low  score. 

'       .  ***** 

I  think  the  C.A.I,  program  is  fantastic.     I  think  the 
program  should  be  spread  through  out  every  school  system  in 
America. 

****** 

Listen  up  and  listen  good..^.I  like  these  computers  becouse 
they  help  you  learn  new  things,  they.... help  you  in  lots  of 
ways.     I  have  math  and  Im  good  at  it  but  I  wish  I  could  be  in 
topics  or  reading.     Now  I  have  lots  of  friends  that  likes 
computers  and  if  you  take  them  away  we  will  be  disapointed. 
So  you.  better  not  take  them  away. 

Data  on  student  attitudes  derived  from  student  questionnaires  were 

evaluated  in  a  sei5arate  study.        In  that  study. a  greater  sense  of 

internal  responsibility  for  success  was  found  among  CAI  students  when 

compared  to  non-CAI  students. 


The  CAI  Coordinators 

The  CAI  coordinators  were  the  most  enthusiastic  supporters  of  the 
CAI  program.     Each  one  had  had  years  of  experience  as  a  classroom  teacher 
and  became  convinced  that  CAI  was  helping  all  students  to  improve  their 
skills.    At  the  conclusion  of  the  study  they  had  many  success  stories  to 
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report  as  well  as  a  few  failures.     Where  failures  were  identified,  they 
were  associated  with  school  failures  in  genieral  rather  than  CAI  in 
particular.     On  the  other  hand,  students  who  were  not  doing  well  in  the 
classroom  occasionally  did  quite  well  in  the  CAI  lab.    As  a  final  statement 
at  the  donclusion  of  the  study  the  CAI  coordinators  agreed  "It's  a  crijne 
to  have  the  CAI  lab  in  use  l^s  than  full  time." 

Implications  for  Schools 

The  ETS-LAUSD  study  has  demonstrated  the  effectiveness  of  the  CCC 
mathematics  reading  and  language  arts  courseware  used  in  a  pull-out  program 
of  drill-and-practice  CAI.  The  effectiveness  has  been  demonstrated  both 
in  one-year  and  longitudinal  studies.     What  does  this  mean  to  school 
administrators  and  school  boards  who  have  to  make  decisions  about  educa- 
tional issues?    Does  this  study  become  a  strong  endorsement  for  computer- 
assisted  instruction?    Why  did  the"  results  in  this  study  occur?  Wherein 
lies  CAI's  effectiveness? 

The  Computer 

The  effectiveness  of  CAI  in  the  Los  Angeles  study  was  dependent 
on  the  computer  only  to  the  extent  that  the  computer  was  a  reliable  in- 
strument for  disseminating  the  CCC  software.     The  computers  used  in  this 
study  were  minicomputers,  dedicated  to  running  the  CCC  software.^  They 
are/were  old-fashioned,  outmoded,  no  longer  sold  through  CCC.  Nevertheless 
despite  the  fact  that  they  contained  none  of  the  accoutrements  of  newer 
and  more  sophisticated  technology,  they  worked.     And  that  is  all  that  was 
necessary  on  the  part  of  the  computers  in^this  study — that  they  should  be 
reliable  instruments  directed  by  the  CCC  software. 


The  CCC  Software 

The  success  of  CAI  in  the  Los  Angeles  study  was  in  a  large  part 
due  to  the  CCC  software.     It  was  the  courseware  and  the  software  necessary 
to  manage  the  courseware  that  determined  what  students  saw  and  responded 
to  while  they  were  in  the  CAI  labs\    Criticisms  of  the  CCC  software — that 
it,  too,  was  old-*fashioned  compared  to  the  possibilities  associated  with 
newer  technologies~did  not  and  could  hot  negate  its  effectiveness  as 
demonstrated  in  the  study.    The  Mathematics  Strands  curriculum,  in 
particular,  demonstrated  a  remarkable  power  to  increase  the  distance 
between  users  and  non-*users  over  the  three  yeiars  of  the  longitudinal 
studies. 

The  CAI  Program 

The  success  of  CAI  in  this  study  may  be  due,  in  some  part,  to 
the  way  in  which  CAI  was  implemented.     In  each  of  the  experimental 
schools  a  CAI  lab  was  established,  a  highly  qualified  teacher  was  trained 
to  serve  as  CAI  coordinator  in  the  lab,  and  a  CAI  program  was  implemented 
whereby  students  attended  the  lab  daily  for  one  or  two  10-minute  sessions 
of  CAI.     Standards  of  behavior  in  the  CAI  lab  were  established  early  and 
maintained;  behavior  problems  were  less  apparent  in  the  lab  than  in 
general.     The  atmosphere  of  the  CAI  labs  was  cheerful  and  work-oriented. 
Tight  operating  schedules  were  developed  in  order  to  accommodate  the  many 
students  and  the  schedules  were  maintained  because  of  student/ teacher 
cooperation.     The  CAI  program  was  a  friendly  and  business-like  enterprise, 
consistently  well  run.     The  CAI  coordinators  and  teaching  assistants — and 
any  adult  who  was  present  in  the  CAI  lab — helped  answer  students'  questions. 
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Motlvatlon  was  enhanced  by  contests  among  classrooms  for  the  highest 
number  of  students  with  80%  or  more  correct  responses. 

Teachers  had  minimal  responsibility  for  the  success  of  the  CAI  program 
but  had  the  opportunity  to  select  the  schedule  which  best  fit  their  classrooms. 
In  the  small  schools  with  only  16  terminals  in  the  CAI  lab,  teachers  remained 
in  their  classrooms  with  half  of  their  students  while  the  other  half  went  to 
the  lab.     Teachers  selected  the  groups  of  students  to  go  to  the  CAI  lab  on 
the  basis  of  the  teacher's  classroom  schedule,  since  the  computer  could 
handle  any  student  in  any  curriculum  at  any  time.    The  same  procedure  was 
followed  with  some  upper  grades  in  the  larger  CAI  schools.    That  arrangement 
provided  teachers  with  twcr  periods  of  time  dally  when  the  teacher/pupil 
ratio  was  half  its  normal  s^ze.     In  ^he  larger  Title  I  school  where  trans- 
portation  to  and  from  the  CAI  lab  might  cause  problems,  teachers  often 
elected  to  bring  the  whole  class  to  the  CAI  lab  at  the  same  time.  Wherever 
it  was  possible ,  teachers  in  grades  1  and  2  were  encouraged  to  bring  the 
whole  class  to  the  lab  so  that  extra  help  could  be  provided  to  very  young 
students.     CAI  schedules  were  made  up  by  the  CAI  coordinator  with  input  and 
cooperation  from  the  classroom  teachers. 

How  can  the  findings  of  the  study  be  generalized?    Are  other  kinds 
of  CAI  likely  to  be  more  or  less  effective?    These  issues  are  a  matter  for 
serious  concern.    Generalizability  of  the  effectiveness  data  is  possible 
only  to  the  extent  that  components  of  the  Los  Angeles  CAI  program  match 
components  of  other  configurations  of  CAI.     In  fact,  many  of  the  components 
of  the  Los  Angeles  CAI  program  can  be  described  in  terms  of  the  findings 
of  school  effectiveness  studies  having  nothing  to  do  with  CAI. 
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Components  of  Effectiveness 

In  mastery-learning  studies,  a  criterion-referenced  test  is  given 
at  the  end  of  each  learning  unit.    The  test  determines  which  students  have 
mastered  the  unit  and  which  need  more  help  to  achieve  mastery. 
Students  provided  with  feedback  and  extra  time  and  help  obtain  the  skills 
necessary  for  subsequent  learning  tasks.     Studies  comparing  mastery- 
learning  classes  with  conventional  classes  have  demonstrated  the  superiority 

13 

of  mastery-learning  (Bloom,  1974).        In  the  CAI  study,  the  software  which 

drives  the  CCC  courseware  determines  the  initial  mastery-level  of  each 

student  in  each  strand  of  the  curriculum,  places  the  student  at  the 

appropriate  entry  level,  and  allows  the  student  to  progre^ss  within  the 

strand  only  upon  mastery  of  material. 

Academic  learning  time  (ALT)  is  defined  as  the  time  a  student  is 

engaged  with  academic  materials  or  activities  that  yield  a  high  success 

rate  wherein  a  student  understands  the  task  and  makes  only  occasional 

errors.     A  major  finding  of  the  Beginning  Teacher-Evaluation  Study, 
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BTES  (Berliner,  1979),  *    was  that  increases  in  ALT  ^re  associated  with 
increases  in  student  achievement.    Because  the  CCC  software  adapts  the 
entry  level  in  the  CAI  curriculum  to  the  ability  level  of  th^  student  and 
allows  the  student  to  progress  only  upon  de;ihonstration  of  mastery,  a 
student  using  the  CCC  curriculums  has  a  high  probability  of  success 
during  each  and  every  CAI  session.  „ 

Early  analyses  from  the  BTES  (McDonald  &  Elias,  1976)^^  indicated 
that  direct  instruction  improved  student  learning.  Direct  instruction 
was  defined  as  having  three  components:  (1)  a  component  that  explains 
what  is  to  be  learned  or  models  it  or  elicits  its  elements  by  questioning 


(italics  ours),  (2)  a  component  that  provides  the  appropriate  conditions 
for  atteiapting  what  is  to  be  learned,  and  (3)  a  component  that  provides 
feedback  on  how  well  the  child  is  learning  the  task.     The  CCC  courseware 
elicts  by  questioning;  the  software  assures  mastery  of  preceding  elements 
before  attempting  further  work;  immediate  feedback  is  given  to  every 
student  response  and,  for  the  benefit  of  both  students  and  teachers,  the 
standing  of  each  student  within  each  strand  of  the  curriculum  is  available 
at  any  time. 

Apart  from  the  CAI  curriculums,  themselves,  the  CAI  labs  demon- 
straj^ed  qualities  that  were  found  to  be  effective  in  other  studies. 
Adaptability  and  consistency  of  instruction  were  directly  related 
to  instructional  efficiency  in  a  study  of  successful  schools  (Vehezky  &  ^ 
Winfield,  1979).^^    Use  of  th^.  CAI  labs  was  a  consistei^t  pattern  for  most 
CAI  students  who  received  10-20  minutes  of  instruction  over  periods  up  to 
4  years.     Although  as  many  as  32  students  were  in  the  CAI  lab  at^any  one 
time,  the  CAI  curriculum  level  covered  a  i)road  range,  adapting  to  the 
ability  levels  of  the  students. 

After  reviewing  others'  research  as  well  as  his' own,  Edmonds  (1979) 
summarized  the  characteristics  of  effective  schools  as  having  (1)  strong 
administrative  leadership,  (2)  climate  of  expectation  of  achievement ,  (3)  an 
orderly  quiet  atmosphere,  neither  rigid  nor  oppressive,  (4)  a  belief  in  the 
importance  of  student  acquisition  of  b^sic  skills,  and  ,.(5)  frequent  monitoring 
of  student  progress.     The  CAI  labs  in.^  the  study  were  run  by  CAI  coordinators 
who  were,  thepselves,  strong  teachers  and  good  program  leaders  who  ran  ' 
the  bright  and  cheerful  labs 'in  a  business-like^  way  and  monitored'  student 
progress  in  basic-skills  areas.  f 


Gage  (1978)      developed  a  set  of  inferences  for  maximizinjg  student  achieve 
ment  in  third  grade  from  a  study  of  the  research. 

.  -     .  '  ....  . . 

-'Teachers  should  have  a  system  of  rules  allowing  pupils  to 
^    attend  to  personal  and  procedural  needs  without  having  to  check 
^with  the  teacher. 

— Teachers  should  move  ground  the  room  a  lot,  monitoring  pupils* 
seatwork  and  communicating  to  their  pupils  an  awareness  of 
behavior  while  attending  to  academic  needs. 

— For  independent  pupil  work,  ^teachers  should  insure  that 
assignments  are  interesting  and  worthwhile  yet  easy  enough 
to  be  completed  by  each  pupil  alone. 

— Teachers  should  keep  .to  a  minimum  activities  such  as  giving 
directions  and  organizing  cla8s  for  instruction. 

— Teachers  should  call  on  a  pupil  by  name  before  asking  a 

question  as  a  means  of  insuring  that  all  pupils  are  given  an 
equal  number  of  opportunities  to  answer  questions. 

— With  less  academically  oriented  pupils,  teachers  should 

always  aim  at  getting  the  child  to  give  some  kind  of  response 
to  a  question. 

--During  reading-group  instruction,  teachers  should  give  a 
maximal  amount  of  brief  feedback  and  provide  fast-paced 
activities  of  the  "drill"  type. 

All  seven  of  the  inferences  describe  the  conditions  in  effect  in  the  CAI 

labs  in  Los  Angeles.'    Although  teachers  did  not  call  upon  students  by  name, 

all  students  were  exposed  to — and  responded  to~many  questions  during 

their  CAl  time.    One  diligent  second-grader  answered  102  questions  in  a 

10-minute  session.     A  tesponse  to  40  or  50  questions  was  more  typical. 

The  CAI  coordinators  moved  about  the  lab,  answering  students*  questions 

and  monitoring  their  work.    Behavior  problems  were  minimal  because  of  the 

system  of  rules  and  expectations  for  behavior  in  the  CAI  lab. 

The  success  of  CAI  in  this  study  may  be  related  to  the  successful 

practices  identified  in  other  effectiveness  studies:    mastery  learning, 


high  academic  learning  time,  ditect  instruction,  adaptability  and  consistency 
of  instruction,  an  orderly  atmosphere  with  expect£^tion  of  success  in 
basic  skills,  the  use  of  drill,  and  equal  opportunity  for  responses  from 
all  students  with  a  high  probability  of  success  in  responding  '  The 
advantage  of  the  computer  for  drill-and-practice  activities  lies  in  the 
computer's  efficient  use  of  time.    For  only  10-20  minutes  daily,  truly 
individualized  drill-and-practice  can  be  used  to  instruct  students  at 
their  own  ability  levels,  to  provide  immediate  feedback  to  each  response, 
to  move  students  ahead  on  the  basis  of  their  mastery  of  subject  matter, 
to  keep  records  of  each  students'  placement  in  each  strand  of  each 
curriculum,  and  to  do  this  with  demonstrable  effectiveness  over  a  period 
of  years. 

CAI  Effectiveness  and  Microcomputers 

The  present  study  produced  its  effects  with  the  use  of  a  CAI  lab 
equipped  with  a  minicomputer  dedicated  to  running  the  CCC  curriculums. 
With  the  tight  budget  constraints  and  iti[iproved  technology  of  the  1980s, 
an  increasing  number  of  school  systems  are  purchasing  microcomputers  for 
computer-assisted  instruction  in  elementary  schools.    Can  the  results  of 
this  study  help  to  predict  the  effectiveness  of  CAI  using  microcomputers? 
Consider,  again,  the  three  major  components  of  the  study: 

The  CAI  lab.    Microcomputers  are  sometimes  used— -one  or  two  at  a 
time — in  a  classroom,  or  in  hallways  or  closets  adjacent  to  the  classroom, 
under  the  superVision  of  the  classroom  teacher.     To  the  extent  that  tnese 
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conditions  prevail,  there  exists  a  very  real  difference  in  the  operation 
of  CAI.     In  the  present  study,  either  the  total  class  received  drill-and^ 
practice  CAI  for  10-20  minutes  in  the  CAI  lab  or  the  teacher  selected  a 
group  of  students  to  attend  the  CAI  lab  at  one  period  of  time  with  the 
rest  of  the  class  attending  at  a  latTer  time.     Teachers  electing  the  later 
option  had  two  periods  of  classroom  time,  usually  back-to-back,  with  a 
considerably  reduced  teacher-pupil  ratio.     In  neither  case  was  the 
classroom  teacher  responsible  for  knowledge  about  the  computer  or  monitoring 
student  use  of  it.    Even  when  microcomputers  are  used  in  a  resource  room 
attended  by  students  needing  remedial  work,  the  conditions  are  usually 
different  in  that  only  one  or  two  students  at  a  time  are  withdrawn  from 
the  classroom.      Where  schools  utilize  multiple  microcomputers  in  a 
network,  or  cluster  configuration,  the  learning  environment  would  be  most 
analogous  to  the  minicomputer  CAI  X^ib. 

The  minicomputer.    Differences  between  the  storage  and  memory 
capability  of  minicomputers  and  microcomputers  are  becoming  less  and  less 
apparent  with  current  technological  advances.    At  present^  however, 
low-cost  mass  storage  devices  tor  microcomputers  are  not^ large  enough  to 
accommodate  a  complete  set  of  CAI  programs  such  as  CCC's  mathematics 
CAI  curriculum  without  customized  adaptation.     The  decreasing  cost  of  the 
microcomputer  and  its  capabilities  with  regard  to  sound  and  graphics 
have  made  it  a  very  attractive  tool  for  education.     However,  in  the  long 
run  it  is  not  the  computer  hardware  but  the  educational  software  which  is 
important  to  computer-assisted  instruction. 

The.CCC  curriculums.    The  major  component  of  any  CAI  program  is  the 
courseware  itself.    What  students  practice — and  what  they  learn — depends 
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on  the  curriculum.     In  this  study,  there  were  differences  among  the^ 

mathematics »  reading  and  language  arts  curriculums  in  the  length  and 

breadth  of  coverage »  accessibility  by  students  of  different  ability 

levels »  and  effectiveness  over  periods  of  one  year  or  more.    Even  greater 

differences  can  be  expected  among  the  myriad  smaller  CAI  programs  developed 

by  hundreds  of  different  authors.    Because  of  the  reduced  storage  capacity 

of  the  microcomputer,  CAI  courseware  similar  to  the'CCC  mathematics  curriculum 

faces  major  changes  when  adapted  for  a  microcomputer.    Those  changes — breaking 

up  the  curric'ulum  into  component  parts  for  example— create  a  CAI  program 

quite  different  from  the  program  evaluated  in  the  current  study.  Microcomputer 

drill-and-practice  courseware  may  or  may  not  be  graphically  more  attractive, 

f 

slower  in  operation,  more  self-paced  and  self  "-selected,  and  more  narrowly  ^ 

focused  within  each  CAI  session.    Whether  the  changes  potentially  increase 

>  / 
or  decrease  the  effectiveness  of  a  mathematics  CAI  program  is  a  question  for/ 

future  research  to  answer.  / 

/ 

/ 

/ 

/ 

■  ■/  ■  ■  / 

/ 
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Chapter  I 

INTRODUCTION  TO  THE  STUDY  .  ^ 

In  July,  1975,  the  Educational  Testing  Service  (ETS)  and  the  Los 
Angeles  Unified  School  District  (LAUSD)  jointly  submitted  a  proposal  to 
theV%tional  Institute  of  Education  (NIE)  to  provide  funding  for  a  study 
6f  the  long-term  effectiveness  of  computer-assisted  instruction  (CAI)  for 
compensatory  education.     Existing  evidence  strongly  suggested  that  CAI  of 
the  type  to  be  studied— drill  and  practice  in  reading,  language  arts, 
and  mathematics — was -effective  for  periods  of  at  least  a  year,  that  its 
costs  were  well  within  typical  compensatory  education  budgets,  and  that, 
unlike  many  other  apparently  effective  compensatory  interventions,  its 
use  could,  easily  be  replicated.    The  ETS/LAUSD  proposal  requested  funds 
to  validate  these  earlier  findings  and  to  examine  whether  CAI's  contribution 
to  student  performance  could  be  maintained  over  periods  as  long"  as  four  years 

Qn  March  15,  1976,  the  NIE  funded  a  research  design  phase  for  this' 
CAI  project  and  on  September  10,  1976,  formally  committed  funds  for  the 
first  year  pf  the  project's  activities.     From  September  1976  through  June 
1980  th6  Los  Angeles  Unified  School  District  operated  four  CAI  labs  under 
a  subcontract  with  ETS.     After  a  final  year  of  data  analysis  and  writing 
at  ETS,  thi  pro ject . ended  in  September  l98l.    This  final  report  is  an 
attempt  to  systematize  what  was  done  and  what  was  learned  during  more 
than  five  years  of  study.     In  this  section  of  the  report  the  focus  will 
not  be  on  the  statistical  analyses  of  the  data,  but  on  a  description  of 


the  project  and  its  impact  on  people.    In  order  to  help  the  reader 
under  stand  the  project  better,  discussion  in  this  chapter. will  focus  on 
the  setting  in  which  the  study  occurred  and  the  CAI  curriculums  which 
were  used. 

The  Setting  for  the  Study 

Several  schools  from  Area  4  of  the  Los  Angeles  Unified  School  District 
were  involved  in  the  Study:     four  as  CAI  schools,  two  as  comparison 
schools,  and  several  others  as  occasional  participants*    Brief  descriptions 
of  six  of  the  schools  follow. 

School  1 

School  1  was  located  one  block  from  the  beachfront,  in  an  area 
containing  shops  and  apartments  of  all  sorts.    One  side  of  the  main 
thoroughfare  boasted  all  of  the  symbols  of  new  affluence  while  the  other 
side  maintained  an  atmosphere  of  counter-cultural  bohemia.    This  school 
was  the  smallest  in  the  study.    The  school  building  was  modern,  small, 
and  neat,  with  touches  of  permanent  art  work  inside  and  out.    The  school 
population  was  primarily  Anglo  and  was  highly  transient. 

During  the  third  and  fourth  years  of  the  study,  a  court-approved 
desegregation  plan  saw  School  1  become  part  of  a  triad  of  schools  in 
which  School  1  served  all  fourth  graders  in  the  triad  and  the  other 
schools"^  served  fifth  graders  or  sixth  gradeis.    The  plan  worked  well  for 


Integration  and  had  both  positive  and  negative  impacts  on  the  CAI  study. 
Longitudinal  data  involving  fifth  and  sixth  graders  at  School  1  were  lost, 
but  two  large  one-year  studies  at  the  fourth-grade  level  were  improved 
because  of  School. I's  larger  population.     School  1  had  no  computer  for 
the  first  three  years  of  the  study  but  was  able  to  operate  a  small  CAI 
lab  as  a  sattelite  ^  the  computer  at  School  2,    During  the  final  1979-80 
school  year,  a  CCC-17  computer  was  used  at  School  1  to  run  an  up-dated 
reading-f or-comprehension  curriculum  together  with  the  older  mathematics 
and  reading  curriculums. 

School  2  - 

School  2  was  al^o  small,  but  larger  than  School  1.     It  was  located 
in  a  well-kept  residential  area.     The  population  it-  served  was  quite 
diverse  economically  and  socially.     The  ethnic  composition  Qf  the  student 
body  was  roughly  half  Hispanic  and  half  Anglo,  with  a  few  others. 
Although  the  student  transiency  rate  was  moderately  high,  there  was  viery 
little  turnover  in  staff.    The  school  building  was  modern  in  design, 
consisting  of  a  number  of  buildings  connected  by  covered  walks. 

Schopl  2  was  largely  unaffected  by  the  LAUSD  desegregation  plan 
since.it  was  already  integrated.     It*s  small  CAI  lab  contained  only  17 

i  ■         ■  ^  '  ■ 

terminals  but  its  computer  served  the  15  terminals  at  School  1  as  well. 
In  addition  to  the  regular  students  involved  in  the  study,  the  CAI  lab  at 
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School  2  served  a  small  population  of  aphasic  youngsters  and  a  group  of  ' 
Hispanic  adults  learning  English* 

School  3  *  o 

School  3  was  very  large.     It  was  situated  in  an  inner  city  o^rea 
a  few  miles  southwest  of  downtoi^m  Los  Angeles.     The  school  was  located  on 
a  short  side  street  between  a  main  surface  road  and  a  freeway.  It 
consisted  of  three  two-story  buildings  connected  by  walkways.     Some  rooms 
had  exterior  doors;  others  opened  onto  hallways;  all  doors  except  the 
''off ice  door  were  kept  locked  at  all  times.     Except  for  a  number  of 
Hispanic  children,  the  school ' population  was  Black.     The  transiency  rate 
among  students  was  extremely  high.     School  3  was  a  Title  I  school. 

During  the  last  two  years  of  the  study,  the  LAUSD  desegregation^ 
plan  provided  'that  50  fifth  and  sixth  graders  ftom  School  3  attend  a 
mid-site  school,  together  with  students  from  the  Valley.     The  plan  was 
successful  in  integrating  a  few  students  from  School  3-^,  but  the  school 
itself  remained  racially  isolated.     The  plan  caused  a  loss  to  the  study 
of  students  in  both  longitudinal  CAI  cohorfs  and  cohort  control  groups. 
School  3  had  a  full-size  CAI  lab  with  a  computer  and  32  terminals. 

School  4 

School  4  was  a  Title  I  school  located  in  an  older  residential  area. 
The  school  building  was  of  the  traditional  style— ^a  two  story  structure 
with  wide  interior  hallways.     The  student  population  was  ethnically 
mixed,  predominantly  minority  with  a  growing  number  of  Hispanic  students. 
The  school  had  a  large  CAI  lab  with  a  computer  and  32  terminals. 

ERIC 
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School  4  was  untouched  by  the  LAUSD  desegregation  plan  during  the 
years  of  the  CAI  study.     It  remained  a  racially  isolated  school.  Complica** 
tions  in  testing  the  Hispanic  population  in  the  final  years  of  the  study 
together  with  a  high  transiency  rate  reduced  the  number  of  students 
available  for  longitudinal  studies. 

School  5 

School  5  was  located  on  a  main  traffic  thoroughfare.    The  stores 
across  the  street  from  the  school  had  their  front  doors  locked  and  could 
be  entered  only  from  the  parking  lot\^:n*-back.    The  school  itself  was 
entered  from  a  side  street.    The  main  school  building  was  old,  designed 
like  a  Spanish  mission,  and  temporary  classroom  buildings  were  situated 
in  the  schoolyard.    The  halls  were  decorated  with  students'  art  and  craft 
work.    Approximately  two-thirds  of  the  students  were  Hispanic  and  one-third 
were  Black.     School  5  was  a  Title  I  school.     It  contained  no  CAI  lab  but 
its  students  participated  in  the  study's  testing  program.     It  was  a 
comparison  school. 

School  6 

School  6  was  situated  on  a  quiet  street  in  a  very  neat  residential 
area.    It  was  a  large  school,  modern  in  design,  consisting  of  a  number 
of  buildings  along  three  sides  of  a  large  school  yard.    The  school 
population  was  predominanatly  Hispanic,  but  a    great  variety  of  ethnic 
backgrounds  were  represented,  including  more  Oriental  children  than  in 
any  other  school  in  the  study.    School  6  lost  its  Title  I  standing 


\ 

■\ 


just  before  the  CAI  study  began.     It  was  a  comparison  school  and  had 
no  CAI  lab. 

CAI  Curriculums 

In  searching  for  CAI  curriculums  it  became  obvious  that  there  were 
no  reading  or  language  curriculums  immediately  available  which  could  be  * 
offered  to  students  across  grade  levels  1-6,     Computer  Curriculum  Corporation 
(CCC)  could  provide  Reading,  Grades  3-6  and  Language  Arts,  Grades  3-6 > 
but  lack  of  audio  signkls  on  their  systems  precluded  offering  a  curricu- 
lum for  beginning  readers.    CCC  could  also  provide  Mathematics  Strands > 
Grades  1-6  in  the  same  software  package  as  their  reading  and  language 
programs.    The  decision  was  made  to  use  these  off-the-shelf  curriculums 
for  several  reasons: 

(1)  They  were  CAI  curriculums  which  were  immediately  available, 
having  had  fairly  wide  usage  in  systems  located  across  the 
United  States. 

(2)  The  CCC  curriculums~especially  Mathematics  Strands— had  a 

body  of  evaluation  studies  which,  although  they  had  shortcomings, 
appeared  to  indicate  success  in  raising  test  scores. 

(3)  CCC  was  the  only  vendor  which  could  offer  all  three  curriculums 
in  one  software  package,  at  a  price  the  study  could  afford. 
Having  one  software  package  assured  the  study  of  a  minimum 
amount  of  time  to  get  the  CAI  operations  running. 

The  CCC  curriculums  are  drill  and  practice  curriculums  which  are  not 
intended  to  teach  students  but  to  reinforce  the  skills  they  have  already 


/ 
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been  taught  in  the  classroom.    A  description  of  each  of  the  curriculums 
follows.  '  I 

Mathematics  Strands.  Grades  1^6  I 

t 

Mathematics  Strands,  Grades  1-6  is  one  of  the  xLosx.  highly  individual- 
ized CAI  (computer-assisted  instructional)  programs  ever  developed.  A 
student  participating  in  the  Mathematics  Strands/program  receives  lessons 
that  are  prepared  for  her  on  the  basis  of  her  own  achievement  and  educa- 
tional needs.     Her  lessons  are  not  stored  in  the  computer's  memory  but 
are  generated  by  the  computer  as  she  works  at  a  terminal.     Because  the 
computer  immediately  checks  the  student's  ^response  to  each  item,  it  can 
adjust  the  lesson's  difficulty  level  whi^4  the  lesson  is  in  progress. 
Furthermore,  it  can  make  this  adjustment  in  each  concept  area  on  which 
the  student  Is  working.     Such  individuializing  capability  represents  a 
significant  step  forward  "in  the  deve^pment  of  curriculum,  material  that 

meets  the  goals  of  individualized  i/istructiOn. 

/ 

oAll  the  topics  in  elementary /school  mathematics,  with  the  exception 
of  geometry  and  word  problems,  a/e  included  in  the  Mathematics  Strands 
program.     Its  stress  on  basic  ^mputational  skil^^makes  it  compatible 

with  a  wide  range  of  textbook/ series.     Because  it  does  provide  individual- 

/ 

ization,  the  program  is  appr^opriate  for  both  remedial  and  accelerated 

classes.  "  / 

The  Mathematics  Strands  program  achieves  its  goal  of  individualized 

instruction  by  using  a  strands  structure.    There  are  fourteen  strands, 

one  for  each  concept  area  included  in  the  program.    The  strands  are: 


Strand  1 

Strand  2 

Strand  3 

Strand  4 

Strand  5 

Strand  6 


Number  Concepts 
Horizontal  Addition 
Horizontal  Subtraction 
Vertical  Addition 
Vertical  Subtraction 
Equations 
Measurement 

Horizontal  Multiplication 

Laws  of  Arithmetic 

Vertical  Multiplication 

Division 

Fractions 

Decimals 

Negative  Numbers 


Strand  7 

Strand  8 

Strand  9 

Strand  10 

Strand  11 

Strand  12 

Strand  13 

Strand  14 


During  every  session,  the  student  receives  a  mixture  of  items  from 
all  the  strands  that  have  exercises  at  her  grade  level.     The  student's 
work  in  each  of  the  fourteen  strands  is  individualized  to  meet  her 
educational  needs.     In  addition,  the  computer  adjusts  the  proportion  of 
exercises  from  each  strand  to  match  the  proportion  of  exercises-  covering 


that  concept  in  an  average  textbook. 

Reading,  Grades  3-6 

Reading,  Grades  3-6  offers  a  supplemental  reading  program  with  two 
important  features:    a  high  degree  of  individualization,  and  a  means  of 
diagnosing  class  and  individual  reading  weaknesses.     Both  these  features 
make  the  program  a  useful  tool  in  building  toward  the  goal  of  individual- 
ized instruction. 

The  program  consists  of  practice  items  designed  to  sharpen  the 
student's  reading  skills  in  five  areas:     word  analysis,  vocabulary 
extension,  comprehension  of  sentence  structure,  interpretation  of  written 
material,  and  development  of  study  skills.     It  contains  enough  material 
for  four  years  of  work,  at  grade  levels  3,  4,  5,  and  6,  as  well  as  supple- 
mentary remedial  material  that  extends  to  grade  level  2.5. 


/ 


Each  student  moves  through  the  program  at  his  ovm  pace.    The  difficulty 
of  the  material  he  receives  is  tailored  to  his  ovm  achievement  level  and 
is  not  affected  by  the  performance  of  other  students  in  the  class.     If  a 
student  needs  remedial  work,  the  program  moves  him  to  a  lower  grade 
level.     If  a  student  needs  to  advance  to  more  challenging  material,  the 
program  moves  him  forward  rapidly.    The  students  in  a  given  class  may 
spread  in  grade  placement  over  every  grade  year  the  program  covers. 

The  program  uses  a  strands  structure  to  individualize  each  student's 


lessons.  Each  of  the  five  skill  areas  the  program  includes  is  represented 
by  a  strand,  or  graduated  sequence  of  related  items.    The  strands  are: 


A  student  who  is  doing  very  well  in  one  area  moves  forward  rapidly 
in  the  strand  which  contains  items  from  that  area.    One  who  is  performing 
poorly  in  one  area  is  moved  back  in  that  stran'.  until  a  level  is  reached 
"that  is  suited  to  the  student's  abilities.    At  this  point  forward  motion 
begins  again.     TYie  rate  of  movement  in  each  strand  is  not  affected  by 
position  or  rate  of  movement  in  the  other  strands. 

The  program  was  designed  with  low  reading  levels  in  mind.     It  begins 
with  very  simple  vocabulary  and  adds  words  from  carefully  selected 
vocabulary  lists.    The  lists  concentrate  on  words  that  children  encounter 
in  reading  materials  and  daily  life  situations. 

A  special  section  at  the  lowest  levels  of  the  program  contains  basic 
two-  to  five-word  sentences.     It  is'  included  as  remedial  work  for  those 
students  in  any  school  grade  who  may  need  it.    The  material  in  this 
section  is  not  differentiated  into  strands.  , 


Strand  A 
Strand  B 
Strand  C 
Strand  D 
Strand  E 


Word  Attack 
Vocabulary 

Literal  Comprehension 
Interpretive  Comprehension 
Work-Study  Skills 
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Language  Arts,  Grades  3-^6 

Language  Arts,  Grades  3-6  attacks  today's  most  common  language  usage 
problems.     It  covers  grades  three  through  six  with  enough  material  for  a 
year's  work  at  each  grade  level.     In  addition,  it  offers  supplementary'^ 
material  designed  for  students  with  special  language  problems:  hearing- 
impaired  students  and  students  for  whom  English  is  a  second  language. 

Language  Arts,  Grades  3-6  supplements  almost  any  language  arts' 
textbook  or  teaching  method.     It  stresses  usage  instead  of  grammar  and 
uses  very  few  grammatical  terms.     Students  using  the  program  learn  by 
example,  pattern,  and  practice. 

The  program  has  two  sections,  strands  and  topics.     In  the  first  year 
of  this  study,  only  the  strands  section  was  used.     The  strands  section 
supplies  individualized  drill  and  practice  tailored  to  each  student's 

achievement  level.     In  the  topics  section,  students  receive  lessons  on  a 

♦ 

topic  assigned  by  their  teacher.  y 

The  strands  section  consists  of  eight  strands,  or  strings  of  items. 


Each  strand  covers  grades  three  through  six.  /  The  progVam  keeps  recotds 

/  \ 

of  each  student's  performance  in  every  strand  and  uses  this  information 

to  adjust  the  studentrs  level  in  each  strand.*  \ 

.       /         THE  STRANDS  /  \ 

Strand  A  »        Pri^hcipal,  Parts  \ 

Strand  B  Verb  Usage 

Strand  C  Subject-Verb  Agreament 

Strand  D  Pronoun  Usage 

Strand  E  ,  Contractions,  Possessives,  and  Negatives 

Strand  F  Modifiers 

Strand  G  Sentence  Structure 
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Summary 

Chapter  I  described  the  six  Los  Angeles  schools  most  relevant  to  the 
study  together  with  some  of  the  forces  which  Impacted  on  the  schools  over 
the  life  of  the  project.    Also  described  were  the  three  drlll~and-prac4:lce 

CAI  currlculums  used  In  the  study:     Computer  Curriculum  Corporation's 

/ 

Mathematics  Strands >  Reading  and  Language  Arts>    .We  now  turn  to  the 
acquisition    and  operation  of  the  CAI  systems. 
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Chapter  II 

ACQUISITION  AND  OPERATION  OF  THE  CAI  SYSTEMS 

This  chapter  will  describe  how  the  three  CAI  systems  were  acquired, 
what  problems  existed,  how  the  CAI  coordinators  were  trained  and,  finally, 
how  the  CAI  program  worked.    Main  topics  are  (1)  early  preparations,  (2) 
the  initial  shakedown  period,  and  (3)  routine  operations. 

Early  Preparations  '  ^  ^  - 

The  Los  Angeles  Unified  School  District  entered  into  a  subcontract 
with  Educational  Testing  Service  in  the  latter  half  of  September  1976. 
Even  before  the  subcontract  was  signed,  LAUSD  was  engaged  in  project 
activities  on  two  fronts:  summer  training  for  coordinators  and  plans  for 
building  modifications.    With  the  signing  of  the  subcontract,  the  proce- 
dures leading  to  the  acquisition  of  CAI  equipment  were  initiated. 

Summer  Training 

From  June  21  through  August  6,  1976,  the  ETS  project  director 
attended  the  summer  training  workshop  for  CAI  coordinators  and  their 
alternates.    Work  accomplished  during  the  workshop  included: 

Observations.     Coordinators  and  alternates  were  able  to  observe 
students  working  with  the  CCC  curriculums  and  supervisory  personnel 
performing  many  of  their  day-to-day  responsibilities  in  three  schools  in 
the  Los  Angeles  area. 

Hands^on-expericnce.  Coordinators  and  alternates  were  entered  and 
enrolled  in  the  Mathematics,  Reading  and  Language  Arts  curriculums  from 
CCC.    The  coordinators  and  alternates  spent  major  portions  of  several 
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days  becoming  acquainted  with  those  curriculums  both  through  hands-on 
,  experience  and  reference  to  the  manuals  for  the  curriculums. 

Preparation  of  materials.    The  ETS  project  director,  the  LAUSD 
professional  expert,  the  coordinators  and  alternates  worked  on  the 
preparation  of  several  products  for  use  in  the  project.    As  part  of  their 
hands-on  experience,  the  coordinators  prepared  progress  tests  at  six 
levels  of  difficulty  in  mathematics  and  four  levels  of  difficulty  in 
reading  and  language.    These  tests  provided  the  basis  for  the  curriculum- 
specific  tests  recommended  by  the  Advisory  Panel  for  administration 
to  all  students  in  the  project.    Coordinators  and  alternates  also  prepared 
materials  for  a  project  handbook  containing  information  on  the  research 
design,  project  activities,  suggestions  for  workshops,  schedules  of 
classes,  random  assignment  of  students  to  treatments  and  other  topics 
relevant  to  the  success  of  the  study.     Finally,  they  produced  material 
for  handouts  to  parents,  teachers  and  visitors. 

The  summer  workshop  helped  to  develop  strong  rapport  among  the 
participants,  increasing  both  the  chances  of  successful  adherence  to  the 
research  design  and  the  enjoyment  by  the  participants  of  their  roles  in 
the  project. 

Building  Modifications 

Blueprints  for  building  modifications  were  begun  in  the  summer  of 
1976  according  to  sketches  in  Appendix  G  of  the  Research  Design  presented 
to  NIE  and  the  Advisory  Panel  in  May,  1976.     Before  the  summer  was  over, 
changes  in  the  building  plans  were  requested.    Specifically,  the  coordina- 
tors asked  that  computer  rooms  NOT  be  subdivided  by  walls  into  three 


sections  as  planned,  but  that'  the  openness  of  the  classrooms  be  preserved. 
Again,  the  plans  for  overhead  room  air  conditioners  were  revised  to 
window  air  conditioners  when  installation  time  was  considered  a  major 
factor  in  delaying  the  start  of  the  study.     Implementation    of  the 
final  blueprint  plans  began  in  the  fall. 

LAUSD  personnel  sealed  doors  from  the  computer  rooms  to  adjoining  class- 
rooms.   They  removed  unnecessary  bulletin  and  chalk  boards  and  removed  a 
sink  in  one  classroom.     They  moved  cabinets  and  installed  wooden  counters 
on  which  the  CAI  terminals  would  be  placed.    They  wired  the  computer  rooms 
and  "Suiit  enclosures  for  the  new  wiring.    They  modified  windows  for  air 
conditioners  and  installed  two  window  air  conditioners  in  each  room  contain- 
ing an  A-16  computer.     Finall]^  they  painted  the  CAI  rooms  and  varnished 
the  furniture.    This  work  was  completed  by  November  15,  and  th6  rooms  were 
ready  at  that  time  to  receive  the  CAI  equipment.     Further  work  would-  be 
necessary  (e.g.,  installation  of  intrusion  alarms,  phones,  etc.),  but  the 
majdr  changes  prerequisite  to  tne  onset  of  CAI  in  the  schools  were  completed. 

Purchase  and  Installation  of  Equipment 

Bidding  for  the  purchase  of  CAI  equipment  could  not  begin  until  the 
contract  between  NIE  and  ETS  and  the  subcontract  between  EfS  and  LAUSD 
were  signed.     At  that  point  the  bids  were  let  in  three  parts:    one  for 
the  purchase  of  three  computers,  one  for  the  purchase  of  100  terminals 
and  four  printers,  and  one  for  communications  equipment  (modems  and  multi- 
plexors) for  use  between  School  2  and  the  satellite  school.  School  1. 


Part  !•     Computer  Curriculum  Corporation  (CCC)  won  the  first  bid  and 
supplied  three  A-16  computers  to  LAUSD,     Delivery  of  the  three  computers 
occurred  between  November  19  and  23\\  Four  terminals  were  loaned  to  the 
schools,  and  an  in-service  for  coordinators  was  conducted  by  CCC  the 
following  week.  '  CCC  also  supplied  the  sof  tware  to  deliver  the  Mathematics 
Strands,  Reading  and  Language  Arts  curriculums.     CCC's  successful  bid 
reflected  the  following  costs:  ^. 

'  Hardware:    3  A"16  computers 

Software;    Mathematics  ^StrandS; 
^  ^  '  Reading 


$20A,360, 
$214 /mo  X  3 
$2l3/mo  y3 


I  Language  Arts      $2l3/mo  x  3 

•  $19*20  X  12  months^  23,040, 
Tax  (6Z)         ,  ^        /    ^  13,644, 

Installation  (maximum)  /  9,000, 

Part  II,     The  David  Jamison  Carlyle  Corporation  v^on  the  bid  to 

sufj^ly  100  terminals  and  four  printers.     L^SD,  by  law,  is  required  to  work 

in  a  competitive  bidding  environment  whi^h  allows  for  practically  no 

deviation  from  published  procurement  conditions.    Unfortunately,  two 

protests  were  filed  on  earlier  terminal  bids  based  on  minor  technicalities. 

Rather  than  risk  legal  action  with  its  attendant  lengthy  delays,  it  was 

decided  to  rebid  the  terminal  procurements.     Delivery  of  50  terminals 

was  contractually  called  for/before  year-end,  and  the  remainder  were  to  be 

available  prior  to  January^ 10,     In  fact,  all  terminals  were  delivered  by 

the  end  of  December,     The  acceptable  bid  reflected  the  following  costs: 


100  terminals 
4  printers 
Tax  (6%) 
Delivery 


$143,995,00 
7,799,80 
9,107^69 
1,800,00 


.6'3 
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Part  111.    Computer  Curriculum  Corporation  won  the  bid  for  the 
communications  equipment.    Their  bid  reflected  the  following  costs: 

2  Modems  $4,710.00 

2  Multiplexors  7,550.00 

Tax  (6%)  735.60 

Installation  200.00 

Modems  and  multiplexors  were  delivered  to  Schools  1  and  2  just  before 
Christmas  vacation.    Noise  on  the  phone  lines  between  the  satellite 
school^  and  the  computer  caused  comiaunication  problems.    After  several 
consultations  with  General  Telephone  personnel,  the  problems  appeared  to 
be  resolved  January  12. 

All  systems  were  up  and  running  the  week  of  January  10,  and  students 
in  all  four  schools  started  computer-assisted  instruction  that  week.  It 
was  a  false  start  for  students  at  Schools  1  and  2,  however,  since  start- 
up problems  plagued  those  schools  especially.    The  next  section  covers 
some  of  the  difficulties  encountered  in  the  initial  shakedown  period. 

The  Initial  Shakedown  Period 

Conventional  wisdom  leads  one  to  expect  problems  in  setting  up  a  CAI 
system.   :,A  small  amount  of  research  into  problems  encountered  by  others 
using  the  CCC  systems  had  encouraged  project  personnel  to  believe  those 
problems  might  not  exist.    Experience  gave  us  mixed  results  with  the 
project's  three  systems. 

System  Problems 

Schools  1  and  2.    The  computer  in  School  2  ran  the  CAI  operations  in 
School  1,  a  satellite  school,  as  well  as  CAI  operations  in  its  own 
school.    Modems  and  multiplexors  in  each  school  allowed  the  CAI  system  to 
operate  over  phone  lines  between  the  schools.     School  1  operated  with  14 

0 
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terminals;  School  2  with  17.     The  situation  in  these  schools  during  the 
initial  shakedown  period  can  best  be  described  in  the  words  of  one  of  the 
coordinators; 

The  sorrows  of  start-up  begin  with  not  starting  up.    One  day 
of  real  work  with  children  and  the  computer  goes  down. 
Mechanical  problems,  unanticipated  and  seemingly  unavoidable, 
keep  recurring.     There  is  nothing  sadder  than  the  face  of 
a  child  whose  number  and  name  have  been  rejected  by  the 
computer . 

The  real  probl^  seems  to  be  that  there  is  no  one 
person  ready  to  assume  the  responsibility  for  getting  the  . 
bugs  out.    You  call  CCC  who  suggests  calling  Cincinnati 
Millicron  who  suggests  calling  Hazeltine  who  suggests 
calling  General  Telephone  who  suggests...  etc.,  etc.  All 
are  very  polite,  very  sympathetic — and  very  ineffective. 

A  new  computer  was  shipped  to  School  2  and  remained  there,  crated 
but  ready,  for  several  weeks.     By  mid-February,  when  school  personnel 
were  just  about  ready  to  give  up  hope-*-and,  indeed,  just  at  the  time  the 
system  was  to  be  inspected  by  the  funding  agency  and  the  Advisory  Panel 
for  the  study—the  CAI  system  began^to  operate  with  its  customary  good 
behavior.     School  2  had  few  problems  after  February,  but  School  1,  the 
satellite,  experienced  some  interference  due  to  noise  on  the  phone  lines. 

School  3.     The  CAI  system  at  School  3  experienced  the  least  difficulty. 
The  A-16  computer  was  delivered  before  Christmas;  the  terminals  arrived 
during  Christmas  vacation.     The  system  was  up  and  running  by  January  3 
with  only  minor  work  needed  on  a  few  terminals.     School  3  waited  one  week 
for  the  other         systems  to  be  ready,  then  began  the  CAIpVc^ram  on 
January  10.    The  computer  at  School  3  was  the  first  to  pass  an  acceptance 
test  and  continued  its  good  performance  through  the  end  of  the  year. 

School  4.     The  CAI  system  at  School  4  was  only  slightly  more  trouble- 
some than  the  system  at  School  3  when  it  was  set  up.    The  CAI  program  was 
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Initiated  on  January  10,  only  one  week  after  the  end  of  Christmas  vacation. 
However,  minor  problems  kept  interfering  with  a  smooth  implementation  of 
the  program.     Late  in  January  the  problems  increased  and  continued 
through  mid-February  when  a  gigantic  effort  by  all  vendors  produced 
results*     Subsequent  problems  were  minimal.    The  coordinator  at  School  A 
kept  a  log  of  her  experiences  during  the  first  year  of  the  study*  The 
log  reflects  the  joys  and  frustrations  of  implementing  the  CAI  program  in 
her  school*     The  log  is  included  as  Appendix  A-* 

Initial  Acceptance 

The  advent  of  a  new  program^  in  any  school  is  likely  to  get  a  mixed 
reception*     In  principle,  the  idea  of  CA^I  was  accepted  whole-heartedly  for 
the  most  part*    On  the  other  hand,  the  ptogram  called  for  several  hours 
of  pretesting — never  a  favorite  occupation  for  teachers  or  students* 
Meetings  were  held  at  each  school, to  explain  the  project  and  the  CAI 
program  to  teachers  and  parents*     The  CAI  coordinators  accepted  the 
responsibility  for  public  relations  activities  and  did  a  remarkable 
job  in  having  the  program  accepted*    They  worked  hard  to  prepare  in- 
service  training  for  teachers,  to  give  them  hands-on  experience  with  the 
computer,  and  to  provide  opportunities  for  teachers  to  observe  their 
students  on  CAI*     There  were  some  complaints,  however,  and  these  revolved 
around  the  issues  of^^^pendability ,  interference,  and  effectiveness* 

Dependability*     The^ceptance  of  the  CAI  program  was  certainly 
related  to  the  dependability  of  the  system*     In  those  schools  which 
experienced  little  difficulty  starting  tip  the  system,  teachers*  attitudes 
were  more  accepting  than  in  the  schools  where  teachers'  expectations 
about  the  CAI  schedule  were  constantly  thwaried*    At  the  end  of  January, 


-19- 

1977;  the  morale  of  coordinators  and  teachers  in  three  of  the  four  CAI 
schools  was  very  low.     By  the  end  of  February,  however,  when  the  systems 
were  operating  with  a  minimum  of  difficulty,  morale  improved. 

Interference.    At  the  time  when  CAI  was  introduced  into  the  schools, 
teachers  had  well  established  classroom  [schedules.     Even  though  the  CAI 
program  had  been  scheduled  early  in  the  school  year,  its  midyear  implemen- 
tation caused  shock  waves  in  familiar  classroom  routines.    After  adjusting  . 
to  the  new  program,  most  fourth'  and  sixth-grade  teachers  were  enthusiastic. 
They  even  looked  forward  to  the  two  periods  in  the  day  when  the  teacher/ 
pupil  ratio  was  reduced.    With  half  the  class  at  CAI,  remaining  students 
could  get  more  individualized  instruction.     For  some  teachers,  however, 
the  CAI  schedule  proved  to  be  an  interference  rather  than  an  aid.  This 
was  particularly  true  for  second-"grade  teachers. 

Since  second-grade  students  had  been  randomly  assigned  to  greater  or 

o 

lesser  amounts  of  Math  CAI,  treatment  groups  had  been  created  which  had 
no  relationship  to  the  groupings  within  the  classroom.    The  CAI  group 
receiving  10  minutes  (later,  seven  minutes)  of  CAI  was  not  out  of  the 
classroom  long  enough  for  the  teacher  to  accomplish  anything  with  the 
remaining  students.     Only  teachers  of  mixed  grades  (1-2  or  2-3) 
reported  little  difficulty  with  the  second-grade  schedule.    One  school 
solved  the  problem  by  having  the  second-grade  teachers  come  to  the  CAI 
room  with  their  students,  returning  to  the  classroom  with  the  group 
that  finished  first.     Two  teachers  who  had  been  opposed  to  the  program 
became  staunch  supporters  under  the  new  schedule. 
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Ef f ectlveness>    Some  teathers  regret  the  existence  of  ever-increasing 
numbers  of  pull-out  programs  which  seem  to  interfere  with  their  job  of 
teaching*     Such  teachers  can  be  won  over  to  a  program  only  by  awareness 
of  the  program's  effectiveness.     Feedback  from  ETS  on  student  progress 
within  the  CAI  curriculums  increased  the  enthusiasm  of  soms  teachers  for 
CAI  and  also  helped  to  identify  students  with  difficulties. 

Routine  Operations 

After  the  initial  shakedown  period,  the  CAI  program  developed  into  a 
smoothly  running  operation.    With  a  minimum  of  system  problems,  coordinatoi^s 
and  aides  were  free  to  improve  schedules,  help  students,  provide  information 
to  teachers  and  develop  routines  for  dealing  with  computer  room  tasks. 
Students  settled  into  the  system  also.     The  final  sections  of  this 
chapter  deal  with  the  routine  of  the  CAI  room.  ^ 

The  Coordinator's  Day 

The  coordinator  and  her  teaching  assistants  shared  tasks  such  as 
housekeeping,  care  of  the  computer  and  terminals,  helping  students,  and 
interaction  with  teachers.    At  the  beginning  of  the  day  the  computer  had 
to  be  turned  on  and  reloaded  and  the  date  advanced.     Terminals  also  were 
turned  on  and  made  ready  for  students. 

As  the  computer  room  day  began,  students  arrived  on  their  own  or 
escorted  by  the  teaching  assistants.    When  students  received  20  minutes 
of  CAI — usually  those  in  the  upper  grades'—every  25  minutes  or  so  a  new 
group  of  students  arrived  in  the  lab.    Younger  students  arrived  every  15 
minutes.    While  students  were  at  the  terminals,  CAI  personnel  responded 
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to  repeated  requests  for  help  with  problems  and  aifswers  to  questions* 
Attendance  was  taken.    Sometimes  notes  were  taken  for  teachers  regarding 
help  needed  by  individual  students.     Incentives  were  developed  to  motivate 
students  to  better  performance* 

At  the  end  of  the  day,  new  students  were  enrolled,  a  backup  tape  was 
run,  teacher  reports  were  prepared  and,  finally,  the  computer  was  shut  down* 

The  Student's  Routine 

Studentflf"  learned  very  quickly  how  to  use  the  CAI  terminals*  First 

graders  were  initially  fairly  slow  at  signing  on— taking  a  minute  or  two 

rather  than  a  few  seconds*     After  a  week  or  so  most  of  the  first  graders 

were  able  to  function  efficiently*     Older  students  learned  almost  immedi" 

ately  and,  except  for  forgetting  their  numbers,  were  able  to  sign  on  and 

begin  work  with  no  delay*     A  fourth  grader  describes  the  procedure: 

When  I  go  to  the  computer  room  we  have  to  line  up  at  the  door 
and  wait  for  a  while  and  then  we  go  in*    When  I  firt,  (sic)  get 
in  I  put  in  my  number  and  name  which  is  1174  and  then  a  space  and 
then  my  name*    After  all  that  I  start  working*     I  like  the  computer 
room  because  it  makes  me  learn  more  math,  language,  and  reading 
and  it  makes  me  use  a  pencil  less* 

Students  often  answered  as  many  as  40  or  50  questions  during  one  10-minute^ 

session*     As  soon  as  one  session  was  over,  students  receiving  20  minutes 

of  CAI  signed  in  for  the  second  session*     At  the  end  of  LO  or  20  minutes, 

students  returned  to  their  classrooms*  ^ 

Summary  / 

In  this  chapter  we  have  described  the  early  preparations  for  the  CAI 
project:     summer  training  for  CAI  coordinators,  the  building  modifications 


for  the  CAI  labs,  and  the  purchase  and  installation  of  the  CAI  equipment. 
The  initial  shakedown  period  was  described,  when  CAI  systems  were  experi- 
encing start-up  problems  and  the  initial  acceptance  of  the  programs  was  less 
than  perfect.    Finally,  the  routine  of  the  coordinator's  day  was  presented,' 
together  with  a  description  of  the  student's  activities  in  the  lab.  We 
now  turn  our  attention  to  CAI  from    ac  perspectives  of  those  who  worked 
with  it.  * 


Chapter  III 
PERSPECTIVES  ON  THE  CAI  PROGRAM 


For  four  years  selected  students  in  four  CAI  schools  received  from 
20-100  minutes  of  drill-and-practice  CAI  each  week.    During  that  time  CAI 
coordinators  worked  with  half  of  the  children  in  each  school  each  year. 
For  four  years  teachers  either  sent  their  students  to  the  CAI  lab  or 
brought  them»  and  each  teacher  had  the  opportunity  to  observe  CAI  in 
action.     Each  year  students  and  school  personnel  were  asked  to  report 
about  CAI.     Students »  teachers  and  principals  were  interviewed;  student 
writing  was  requested;  and  coordinators  attended  a  debriefing  conference 
each  year.     In  this  chapter  CAI  will  be  summarized  by  the  people  who 
participated  in  the  study.     Coordixiators  will  describe  successful  and 
unsuccessful  students  in  the  CAI  program.     Students  themselves^jwill  tell 


how  they  felt  about  CAI.  The  results  of  interviews  with  teachers  and 
principals  will  be  presented. 


At  the  end  of  the  study,  the  CAI  coordinators  were  asked  to  write, 
among  other  things,  a  brief  description  of  the  students  they  felt 
represented  their  successes  and  failures  with  CAI.     Their  case  studies 
are  presented  in  this  section.     In  School  1  the  CAI  coordinator  made  the 
following  reports: 

*Thanks  are  due  the  following  people  for  providing  the  information  for 
this  section:     Mary  King,  Marge  Lord,  Judy  Newman,  Ann  Vasilopoulos ,  and 
Rayma  Wells. 


Successes  and  Failures 
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Bruno  did  poor  work  In  his  regular  fourth-grade  classroom  but 
scored  second  highest  In  the  school        the  mathematics  CAI 
curriculum.    The  last  report  showed  his  yearly  gain  to  be  2.1 
years  and  his  average ^placement  at  the  end  of  May  was  5«6. 
Bruno  took  great  pride  in  his  daily  scores  and  his  classmates 
encouraged  him  to  work  hard  to  help  their  team  in  competitions* 
He  asked  for  very  little  help— only  when  a  new  type  of  problem 
appeared  in  his  lesson. 

*  *  *  ie 

Eva  began  CAI  as  a  disinterested  participant.    Her  attendance 
was  poor  and  at  one  point  she  was  away  six  weeks  on  a  trip. 
When  she  returned  she  Just  couldn't  settle  down.    By  having  the 
teaching  assistant  or  the  coordinator  sit  beside  her  and  encourage 
her  to  trust  her  own  Judgment  and  work  a  little  faster  she  begs^  to 
make  a  score  of  over  twenty  correct  responses  in  her  Reading-for- 
Comprehension  CAI  lessons.    Soon  she  was  able  to  make  between  90 
and  100%  with  twenty-five  or  more  problem^  attempted.    By  the  end 
of  May  she  had  gained  1.4  years  with  a  total  of  only  86  sessions. 
The  average  gain  in  the  Reading^f or^Comprehension  curriculum  in 
4th  grade  was  .9.    £va*s  success  inspired  her  younger  sister  to 
improve  her  lessons  also.    Two  happier  little  girls  were  proud  of 
their  CAI  accomplishments  by  the  end  of  school. 


Mike  was  a  fourth  grader^assTg!red^-^o— the_CAI_rea^ 
curriculum.    He  reached  4.5  in  the  first  lO  lessons^^wTttrxapl-d-^. 
motion  but  never  got  past  4.7  the  remainder  of  the  year.  He 
would  look  at  a  question  and  find  every  reason  not  to  read  or 
answer  it.    He  seldom  answered  more  than  lO  or  11  items  in  10 
minutes.    At  one  point  I  had  him  start  over.    The  first  lesson 
he  made  100%  and  then  he  went  right  back  to  his  previous 
pattern  of  slow  motion.    This  child  was  a  behavior  problem  in 
his  regular  classroom  and  on  the  playground.    His  home  consisted 
of  a  young  father  and  his  younger  girl  friend  who  appeared 
indifferent  to  Mike's  progress  or  behavior  in  school. 

*  *  *  * 

The  most  conspicuous  failure  in  four  years  of  CAI  was 
Rajah.    He  was  a  fourth  grader  assigned  to  the  CAI  Reading 
curriculum.    This  child  was  capable  of  doing  satisfactory  work 
but  could  not  accept  the  computer  report  of  less  than  a  perfect 
score.    He  complained  that  the  computer  cheated  him  as  he 
pounded  on  the  terminal  or  ran  away.    He  was  under  the  counter 
or  screaming  half  the  time  he  was  in  the  lab. 


-25- 


The  teaching  assistant  in  the  lab  would  sit  beside  him  and 
try  to  keep  him  calm.    Finally  it  was  decided  his  disruptivehess 
was  unfair  to  the  rest  of  the  class  and  he  was  dropped  from  the 
study.     Many  days  he  tried  to  sneak  in  and  sign  on  for  a  lesson 
but  he  never  gave  up  wanting  a  100%  lesson. 

His  problems  in  all  phases  of  school  life  continued  the 
entire  year.  The  NPI  at  UCLA  is  trying  to  help  this  boy  and 
his  mother  at  the  present  time. 

In  School  2 — a  small  school  with  less  turnover  than  otjier  schools  in 

the  study — the  following  successes  and  failures  were  reported: 

Two  5th-grade  students  topped  out  of  math  this  year.  They 
are  bright,  eager,  achieving  students  in  every  respect  but 
what  makes  one  story  unique  is  that  one  of  the  students  was 
asking  for  a  transfer.    There  had  been  a  spate  of  springtime 
name  calling  and  this  student  had  been  targeted  because  she  is 
markedly  different,  the  only  oriental  in  the  school.  Sympathetic 
support  from  her  teacher,  counseling  by  the  principal,  support 
by  her  peers  had  been  to  no  avail.     She  still  was  determined  to 
go  to  a  school  with  "her  own"  as  she  put  it.    As  educators  we 
know  that  if  she  ran  from  the  problem  it  would  never  really 
get  solved.    There  was  only  one  thing  that  had  not  been  tried — 
with  the  help  of  her  friends  and  her  competitor  in  the  Cf.A.I. 
lab — we  told  her  that  if  she  left,  the  remaining  student  would 
top  out  while  her  chance  would  be  gone.     It  worked.     She  came 
back,  faced  the  problem  and  topped  out  of  math.     She  was  the 
first  student  in  four  years  of  CAI  to  top  out  of  the  math  cur- 
riculum.     Her  competitor  topped  out  A  few  weeks  later.    We  think 
of  CAT  as  an  academic  remedial  program  but  it  does  act  as  a 
challenge  and  motivator  for  high  achievers  as  well. 

*  *  *  * 

One  student  had  been  coming  to  the  CAI  lab  on  math  for 
three  years.    To  say  he  was  bored  is  not  to  say  enough.    He  was 
clever  enough  to  time  the  problems  so  he  would  do  the  absolute 
minimum  and  get  a  percentage  that  would  not  move  him  too 
many  points  higher  but  still  not  incur  the  anger  of  the  CAI 
coordinator.     I  thought  for  a  long  while  that  he  really 
didn't  like  math  and  this  was  his  way  of  getting  by  until  his 
sibling  was  also  enrolled  on  CAI.    Her  response  was  pretty 
identical  to  his.    And  then  I  met  the  parents  at  Open  House  and 
it  became  only  too  clear  that  these  students  were  reflecting 
parental  values,  "machines  aren't  necessary".     *Vhen  I  went  to 
school  all  we  needed  was  a  chalkboard."    '"Why  spend  our  taxes 
this  way  etc. 
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It  would  have  been  a  little  more  acceptable  if  they  lived 
that  way  but  even  a  casual  inquiry  revealed  that  they  had  the 
usual  TV,  washing  machine,  dishwasher,  car,  etc.,  all  the  electro- 
nic helpers  of  our  age.     School,  however,  was  to  stay  unchanged. 
A  19th  century  approach  to  20th  century  problems. 

Let  me  put  it  this  way.    We  are  rapidly  approaching  the 
2lst  century  and  we  must  as  parents,  educators  and  concerned 
citizens  meet  the  needs  of  our  children.    A  literate  child  with 
good  basic  skills  both  in  math  and  communication  is  going  to  be 
able  to  function  and  succeed  at  whatever  career  or  careers  that 
they  choose.    There  is  great  merit  in  exposing  our  children  to 
technological  awareness  in  a  manner  that  they  view  the  machinery 
as  a  tool.    The  CAI  program  exposed  children  to  computers.  They 
are  not  awed— "stupid  computer"  is  a  comment  often  heard,  a  They 
really  look  at  the  computer  as  an  aide,  just  as  they  view  all 
machinery.    They  learn  the  vocabulary  rapidly  "bits  vs  bytes," 
"computer  down,"  "reload,"  etc.  and  they  understand  at  least  the 
concept.    When  a  student  apologized  to  Dr.  Ragosta  for  taking  so 
long  to  complete  a  task  she  said,  "the  mechanic  did  some  repairs 
and  we  dropped  a  day's  memory."    She  knew  what  she  was  saying.  It 
is  a  valuable  experience  for  students  to  be  able  to  understand  the 
present.    How  else  are  they  going  to  build  the  future? 

School  3  is  a  large  Title  I  school  designated  as  a  racially 

isolated  minority  school.    The  population  is  predominan^'tly  Black  with  an 

increasing  percentage  of  Mexican-Americans.    The  coordinator  reported  on 

a  generally  successful  year:  ^' 

A  group  of  ten  5th-grade  boys  and  girls  were  especially  successful 
in  the  CAI  lab  this  year.     They  were  all  assigned  to  the  CAI 
mathematics  curriculum.     I  believe  their  success  was  related  to 
three  factors;  three  of  the  Sth-^grade  teachers  liked  teaching 
math  and  placed  special  emphasis  on  it;  the  children  responded 
enthusiastically  to  running  reports  and  competing  against  their 
individual  progress,  and  finally,  the  children  took  pride  in 
the  challenge  the  curriculum  offered. 

There  were  no  outstanding  failures  at  School  3  this  year. 
The  few  children  who  did  not  succeed  in  CAI  did  not  succeed  in 
the  classroom  either.    One  4th-grade  boy,  assigned  language, 
sometimes  refused  to  do  his  lesson  and  left  the  room  without 
permission.    His  teacher  reported  he  responded  to  frustration 
in  the  classroom  in  the  same  fashion. 
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School  4  is  also  a  Title  I  school  with  a  large  Spanish-speaking 
population.    The  coordinator — new  this  year — used  a  reward  system  of 

c 

issuing  tickets  for  good  performance  at  CAI.    The  tickets  could  be  used 

for,  admission  to  the  "Westworld  Arcade" — the  CAI  lab  supplied  with 

computer  games  and  kept  open  for  an  hour  after  school.     The  following 

successes  and  failures  were  reported  from  School  4. 

At  School  4  one  or  two  transitional  Spanish-speaking  students 
were  able  to  bridge  the  language  hurdle  and  master  the  vocabulary  in 
the  language  curriculum  well  enough  to  complete  lessons  with  a 
degree  of  success.    This  happened  much  to  the  amazement,  of  the 
classroom  teacher  who  was  sure  the  students  must  be  removed  from 
language  and  placed  into  mathematics.     In  the  classroom,  these 
students  were  not  reading  at  all.     One  or  both  of  these  students 
were  able  to  read  aloud  the  problems  and  select  the  correct  answer 
most  of  the  time—without  frequent  help  from  the  teaching  assistant. 

The  CAI  language  curriculum  patterns  may  have  been  learned  by 
these  students  through  repetition.    Perhaps  the  fear  of  negative 
peer  judgment  (lack  of  approval)  or  teacher  disapproval  may  have 
inhibited  these  students  in  the  classroom.     They  may  have  "given  up" 
performing  in  English  in  front  of  other  people.     The  computer  is 
non-verbal  and  much  less  threatening  than  the  teacher  or  other 
students.     Perhaps  that  accounts  for  the  students'  progress. 

*  *  *  * 

Juan,  10  years  old  is  a  5th-grade  student — quiet,  sensitive 
child  who  was  formerly  enrolled  in  a  Special  Day  Class.    He  is 
currently  seeing  the  Mainstream  Resource  Specialist  for  45  minutes 
daily  in  addition  to  the  corrective  P.E.  specialist  weekly.  This 
was  his  third^ year  in  tfie  CAI  lab. 

Juan  began  hanging  around  the  lab,  after  the  first  of  the  year, 
at  any  free-time  periods,  recess,  lunch  and  especially  after  school. 
Because  he  was  assigned  to  the  Reading  and  Language  CAI  curriculums 
and  received  100%  daily  (a  $1.00  ticket  per  perfect  score),  he  was 
one  of  our  earliest  "steady"  customers  in  the  Westworld  Arcade 
(Admission:  $2.00  in  ticket  money).    He  carefully  observed  the 
activities  and  soon  Volunteered  to  lend  a  hand  by  teaching  some  of 
the  computerized  games.    We  found  this  a  great  assist  as  time  was  at 
a  premium  with  20-25  students  in  the  lab  daily  during  the  3:00-4:00 
o'clock  time  slot.     From  this  point  he  moved  on  to  assisting  with 
the  clean-up  af.ter  closing.    Then  he  began  collecting  admissions. 
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He  was  very  adept  at  the  later,  ironically  in  that  he  was  doing 
poorly  in  math  and  was  seeing  the  specialist  for  improvement.  He 
was  extremely  careful  in  collecting  the  correct  change,  often  to 
the  duress  of  his  pe^rs  who  offered  bribes  and  threats  to  gain 
entrance. 

Near  spring  vacation  Juan  had  become  a  regular  CAI  employee. 
He  had  so  carefully  observed  our  back-up  procedures  and  security 
checks  on  closing  that  he  finally  asked  to  perform.    He  thrilled  in 
assuming  responsibility.    Shortly  afterward,  he  was  meeting  our  cars 
oh  arrival,  pacing  back  and  forth  along  the  fence  if  we  were  a 
little  late.    This  ^fae  especially  interesting  when  we  found  he  lived 
farther  from  school  thap  any  other  child  in  walking  distance.  He 
was  officially  in  another  school's  attendance^  zone.    His  dally  walk 
must  have  taken  a  good  thirty  minutes  either  way.    Neither  rain'  nor 
fog  hindered  his  meeting  us.    He  delighted  in  turning  off  the  alarm, 
turning  on  the  machines  and  setting  the  correct  day  and  time.  We 
noticed  a  steady  improvement  in  his  ability  to  st^ate  the  correct 
tlime.    He  always  made  a  point  of  checking  that  he  was  correct  before 
typing  the  numerals.    He  delighted  himself,  as  well  as  us,  setting 
the  military  time  in  the  afternoon  back-up.    Remember,  Juan  had  been 
having  extreme  problems  in  math,,  but  according  to  his  classroom 
teacher  and  the  Resource  Specialist,  has  shown  remarkable  progress. 
He  enjoys  doing  CAI  mathematics  demos  and  does  only  that  at  every 
opportunity.    He  could  easily  operate  the  lab,  answering  the  tele- 
phone, keeping  inventory,  doing  clean-ups,  even  doing  minor-  typing 
jobs.    He  is  usually  the  first  to  notice  a  problem  in  the  machinery, 
often  asking  quite  technical  questions  as  to  their  function.     It  has 
been  a  personal  Joy  and  a  professionally  rewarding  experience 
watching  Juan  progress  through  the  months. .. thanks  to  CAI. 

*  *  *  * 

James,  10  years  old,  is  a  fifth-grade  student  in  a  fifth/ 
sixth  grade  combination  class.    He  is  a  state-identified 
mentally  gifted  minor,  a  highly  volatile  boy  who  finds  it  most 
difficult  adhering  to  school  standards.    He  is  the  only  child 
of  a  mixed  (Japanese/black)  union.    He  often  feels  he  is  being 
singled  out  for  ridicule,  when  often  he  is  guilty  of  the  very 
same  behavior  by  shouting  inflammatory  statements  to  his 
classmates  in  the  lab.  .  His  group  of  eleven  (11)  students 
happens  to  be  one  of  the  smallest  visiting  the  CAI  lab.  The 
majority  of  the  class  is  Spanish-speaking,  he  being  one  of  the 
two  black  males  enrolled.    His  approach  to  the  CAI  was  most 
inconsistent.    He  never  arrived  with  his  class,  nor  left  with 
them.    He  was  strictly  a  loner.    He  felt  no  compulsion  to  do 
make-up  lessons  tior,  to  perform  to  his  greatest  ability.    He  was 
not  challenged  by  motivational  rewards.    His  overall  progress 
though  above  grade. level,  was  held  to  a  minimum  due  to  this 
negligent  attitude.    Whm  this  student  was  absent  remarkable 
change  was  noticeable  in  his  class's  performance.    He  is 
currently  seeing  the  Resource  Specialist  for  academic  assistance. 
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Even  though  the  coordinators  wrote  about  both  successes  and  failures, 
they  were  able  to  think  of  many  more  successes  than  failures  with  computer 
assisted  instruction.    Where  CAI  failures  were  identified,  they  tended  to 
be  associated  with  school  failures  in  general  rather  than  with  CAI  in 
particular.    On  the  other  hand,  students  who  were  no4  doing  w^ll  in  the 
classroom  Occasionally  did  quite  well  in  the  CAI  lab  ar.,      wen  reported 
liking  CAI.  . 

In  general  CAI  cpordinators  perceived  CAI  as  ^helpful  to  students  and 
anticipated  that  CAI  would  become  even  more  helpful  when  teachers  and/ or 
coordinators  ran  the  program "without  the  restrictions  of  the  research 
design.     The  perceived  success  of  the  CAI  program  during  its  four  years  of 
operation  laid  the  foundation  for  the  successful  transition  of  CAI  from  a 
federally-funded  project  to  a  school  program.     In  an  effort  to  help  that 
tran^tion  and  to  perpetuate  the  knowledge  and  expertise  gained  by  CAI 
coordinates,  a  Conference  was'held  at  the  close  of  the  study,  and  coordina 
tors  helped  to  produce  a  booklet  ^entitled  Thoughts  on  a  CAI  Program. 
That  booklet  is  included  as  Appendix  B.^ 

^  Student  Perspectives  '  ' 

Each  year  of  the  study,  studeA^  produced  writing  about  CAI  from 
assignments  in  their  classrooms.     During  the  last  two  years  the  stimulus 
for  the  writing  was  the  following  l<atter: 


\ 
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Dear  Student: 

It  Is  not  often  that  students  in  elementary  school 
get  a  chance  to  influence  decisions  made  by  the  government 
in  Washington.    However,  you  may  be  able  to  influence  the 
government  in  two  ways.    You  know  you  have  been  part  of  a 
federally-funded  project  about  computer-assisted  instruction. 
People  in  Washington  will  read  the  results  of  that  research 
study  and  will  make  decisions  about  the  future  of  CAI. 
They  may  decide  to  help  other  schools  to  get  CAI  or  they 
may  not.    Before  the  government  makes  a  decision,  t.hey 
should  hear  from  you  and  your  teachers.     How  do  you  feel 
about  CAI? 


,  With  several  hundred  letters  to  select  from,  we  have  chosen  36  to 
report.    The  36  are  a  selected  sample  of  the  more  interesting  letters. 
They  are  a  biased  sample  in  the  sense  that  we  have  over-represented 
negative  letters  since  they  help  to  alert  one  to  potential  problems.  In 
general,  65-70  percent  of  the  statements  about  CAI  made  in  any  year  by 
students  were  completely  positive.    Only  10-13  percent  of  statements  made 
in  any  year  were  completely  negative.     The  remaining  statements  were • 
neutral  or  mixed. 

The  children  speak  for  themselves  without  editorial  assistance: 


I  ttiink  the  computer  is  alright.    And  it  does  help  you  but  I  really  dont 
like  it'  that  much.     But  I  think  I  learn  more  from  the  computer,     then  I 
do  from  the  techer. 

*  *  *  * 

I  think  the  computers  are  boring.    You  do  the  same  thing  everyday.  I 
learn  an  awful  lot.     But  I  think  it  is  boring.     Espeacially  when  you 
don't  understand  something. 

*  *  *  * 
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Ac  first  I  want  to  go  to  the  Computer.    And  I  still  like  the  computer 
alittle  but  it  helped  me  very  much  because  at  first  I  did'nt  remember  but 
the  computer  hlep  remember  more  of  what  I'm  doing  and  now  I'm  going  in 
higher  group. 

*  *  *  * 

I  think  that  the  Computer  is  fun  and  it  can  help  you  learn,  and  to  be 
smarter  in  class.    A  computer  is  a  machine  that  give  you  all  the  answers. 
A  terminal  is  something  that  gets  it  questions . from  the  computer  A 
computer  is  a  nice  thing  to  have.     It  helps  you  in  what  you  need  to  work 
on.     And  a  Computer  is  like  a  teacher.     It  asks  you  questions  and  you 
answer  it.     I  think  a  computer  is  more  smarter  than  you  are  and  it  is 
smarter  than  anything. 

*  *  *  * 

I  think  the  computor  is  allmost  the  best  class  in  this  whole  school.  For 
one  reason  it  helps  you  to  understand  things  that  you  dont  understand  and 
it  helps  you  to  know  things  that  you  don't  know.    You  want  to  know.  One 
more  thing  is^  like  the  teacher.    They  are  very  nice  if  you  don't  know 
something  all  you  have  to  do  is  raise  your  hand  and  they  will  help  you. 
I  am  very  happy^hat  we  have  computor  in  our  school.  '  I  wish  that  we 
would  have  got  fl* earlier.  I'm  also  glad  that  we  get  to  go  everyday  of 
the  week.  ' 

*  *  *  * 

I  like  Computer  Lab,  and  then  agian  I  don't. 

I  like  Computer  Lab  for  the  idea  of  working  with  an  important  machine. 
And  I  don't  like  it  because  I  want  to  work  in  math.     I'm  not  very  good  in 
math  and  I  would  like  to  get  better. 

*  *  *  *  . 

The  reason  why  I  like  the  computer  is  because  I  learn  new  words  eveir^day 
and  I  like,  for  instance  like  the  word  "capsize"  it  means  overturn.  And 
it  strecthes  my  reading  and  makes  it  interesting  and  fun!    So  it  really 
interest's  me.    And  each  day  I  go  to  the  Computer  I  study ' the  matter 
first  and  then  type  in  my  answer.    Typing  is  part  of  the  fun!     I  get 
familiar  with  which  keys  to  punch  in  so  I  can  finish  my  lessons,  fast  but 
carefully.     And  most  of  all  it  keeps  me  on  schedule  and  on  time  for 
everything.     Each  terminal  has  a  screen  and  it  looks  like  a  tv  with  a 
typewriter  hooked  on  to  it.    The  boringpart  about  the  computer  is  the 
percentage  like  100%  and  95%  or  80%  or  else  0%.     Some  of  the  times  when  I 
don't  think  about  what  I'm  doing  I  accidentally  pusli  the  dredit  button 
the  computer  counts  it  as  a  mistake.     That's  why  I  like  the  computer.  ?  \. 

•k   -k   -k  -k 


I  like  the  C* A.I •  room  ^ecause  when  I  was ^little  I  got  bored  of  the 
teacher  talking*  I  wished  triat  something  ciould  teach  besides  the  teacher. 
Well  I  got  my  wish  and  I\wanV  to  thank  the\ people  that  let  the  boys  and 
girls  of  (School  A)  to  go\to  Cl.A.I.  room,    yhy  and  please^  let  3rd, ^  1st 
and  5th  grades  go  to  C.A.l\  zoom  because  I  feel  sorry  for  them. 


\ 


•k  -k  Ik 


I  think  the  computer  is  something  ^  look  forward  to.     I  know  some  people 
that  like  the  computer  as  much  ^s  I.\   Sometimes  I  don't  want  to  go 
because  I  am  tired  but  I  go  anywW  because  I  don't  like  class  work*/  I 
like  the  computer  work.  \ 


I  feel  that  the  computer  should  have  a"  buttim  for  wheti  you  want  to  use 
the  restroom. 


k  k  k  it 


\ 


I  think  the  CAI  stink.  It  is  so  boring.  Sometimels  I  put  r^e  rite  thi\ig 
and  it  says  rong.     I  think  the  CAI  should  be  junked) 


k  k  k  it 


Computers  help  me  but  I  dent  like  them.  They  tire  me  out  because  the 
lessons  are  too  long. 


k  k  k  it 


I  like  the  computers  at  our  school.  They  help  me  to  do  my  math.    When  I 
first  came  to  class,  I  did  not  know  to  divide  and  when  I  started  to  go  to 
computer  I  lerned  how  to  divide.     It  helped  me  more  with  Multiplicatia 
too.     If  little  boys  or  girl's  start  when  they  are  small  they  can  learn 
better.    We  can  do  best  if  we  start  when, we  are  like  seven  or  less. 
Thank  you. 


k   k   k  it 


1  don't  feel  that  you  should  take^computer  out  of  school.     For  myself  the 
computer  has  halped  me  a  lot  in  my  skills  of  language  and  reading.  I 
have  improved  a  lot.     (and  I  got  used"  to  coming  to  computer)  some  people 
think  computer  is  boring  but  it's  not.     So  if  you  take  the  CAI  computers 
my  grades  are  £oing  Iqw.    Sincely.    PS.'   Don't  take  the  computers. 


it  k  k  it 


I  like  it  because  when  I  want  to  get  in 
right  position  .and  stop  acting  foolish 
terminal  at  home. 


a  higher  level  I  Just  get  in  my 
I  would  just  enjoy  having  a 


I  like  computer  because  we  get  time  to  get  away  from  Mr.  N« 

*  *  *  * 

I  think  we  should  not  have  CAI  because  families  would  have  to  pay  more 
money*     I  really  don^t  like  CAl  because  I  think  teacher *s  can  teach  more 
in  a  half  hour  than  computors  can  and  also  they  are  uninteresting* 

*  *  *  * 

Computers  is  an  exiting  event,  everybody  is  working  and  trying  hard  to 
get  one  hundred  percent.     It  feels  like  we're  a  great  big  family,  just 
doing  our  jobs,  so  I  like  computers  even  though  sometimes  I  get  a  low 
^core. 

From  your  computer  lover. 

*  *  *  * 

I  think  the  C.A.I,  program  is  fantastic.     I  think  the  program  Should  be 
spread  through  out  every  school  system  in  America. 

P.S.    Dont  you  think  the  computer  should  have  spelling  too. 

*  *  *  * 

Listen  up  and  listen  good.... I  like  these  computers  becouse  they  help  you 
learn  new  things,  they.... help  you  in  lots  of  ways.     I  have  math  and  Im 
good  at  it  but  I  wish  I  could  be  in  topics  or  reading.    Now  I  have  lots 
of  friends  that  likes  computers  and  if  you  take  them  away  we  will  be 
disapointed.     So  you  better  not  take  them  away. 

P.S.     I  fell  sorry  for  you  if  you  take  them  away. 

*  *  *  * 

I  ^el  happy  in  C.A.I*  sometimes  but  I  feel  sad  because  I  ddn*t  get  100%. 
I  already  have  about  twelve  of  them.     I  love  to  come  to  C.A.I,  with  my 
friends  because  we  usually  have  fun^     I  am  in  one  of  the  highest  courses. 
I  topped  out  of  Reading  and  Language  and  now  I  am  10:^  Fractions.  Everyday 
we  go,  you  should  see  us  how  we  run  up  those  stairs;     I  really  like  it 
because  the  way  the  buttons  sound  when  you  press  on  them.     I  do  my  best 
and  I  love  it." 

*  *  *  * 

1  would  like  the  C.A.I,  to  be  changed  in  a  special  way  so  that  everybody 
could  get  a  chance  to  go.  .  I  don't  think  it  is  fair.     I  would  like  the 
C.A.I,  to  be  changed  so  that  everyone  could  have  a  chance  to  work  as  a 
helper  in  the  C.A.I,  if  they  would  like  to.     I  would  like  to  see  the 
report  on  myself  everyday.    That's  how  I  would  like  C.A.I,  to  be  changed. 
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I  like  computers  and  I  don*t  like  them*     The  reason  why  I  like  the 
computers  is  they  helped  me  improve  in  my  work  and  I  became  proud  of 
myself.    And  the  reason  I  don't  like  computers  is  because  they  force  you 
to  concentrate* 

*  *  *  * 

I  like  computer*    Sometimes  I  act  hyper,  in  computer  and  sometimes  I  am 
mad  and  I  don't  do  too  good*    Even  when  I'm  happy  I  do  terrible,  but  when 
I'm  not  happy  or  sad  or  made  I  do  pretty  well*     I'm  trying  to  tell  you 
that  my  work  in  computer  is  how  I  feel*     I  try  to  do  my  best  but,  I  have 
a  babbit  of  showing  off,  I  try  not  t6  but  I  can't  stop,  when  I  get  in 
trouble  in  computer  or  anywher  at  school  I  get  more  in  trouble  when  I  get 
thome,  by  my  dad*    When  people  yell  at  me  in  school  I  don't  think  it's 
right.,  but  after  a  while  I  understand* 

*  *  *  * 

In  computers  my  subject  is  Langage  &  Reading*     I  like  doing  the  work 
except  sometimes  I*m  not  in  the  mood*    but  I  know  its  helping  me  so  I  do 
the  best  I  can*    The  lady  that  helps  us  Mrs*  Lord  is  very  nice  she  does 
charts  of  what  were  doing  and  makes  schedules  and  she  goes  to  other 
schools  to  help  them*     She  puts  all  her  efort  in  doing  things  to  help  us* 
Other  kids  don't  really  relize  that*    Any  how  I  think  you  should  go  on 
with  the  computers  because  it  helps  kids  more  than  they  think  and  it 
teaches  them  how  to  be  better  at  things  that  they  have  trouble  on  and 
helps  them  do  the  work  faster  and  think  faster*    At  my  school  they  make 
games  and  all  the  classrooms  try  to  get  high  scores  in  there  work  so  that 
their  class  can  win  and  get  an  award* 

So  I  think  that  you  shbuld  keep  the  computer  program  going  even  though 
they  don't  like  it*    It  still  make  you  learn  &  think  faster* 

*  *  *  * 

Computers  sometimes  are.  very  boring,  tiring  and  also  no  fun*  But,  then 
again,  we  run  to  C*I*A*  after  lunch  'cause  the  room, is  air  conditioned* 

*  *  *  * 

I  dont  think  computers  are  very  fun  I  know  that  I'm  learning  morri  than  I 
would  in  my  class  aloan  but  I  find  CAI  boring  you  keep  on  getting 
question  after  question  I  take. only  Math^M  and  would  find  CAI  mor 
interesting  if  I  could  do  some  thing  a  little  different  once  and  a  while* 
Truly,  K*  H. 

*  *  *  * 
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I  thought  the  CAI  was  great  because  it  helped  me  improve  in  math.  I 
thought  it  was  a  challenge  because  I  never  get  a  100%  percent, 

*  *  *  * 

\ 

I  liked  computers  because  it  was  a  lot  of  fun.     I  had  a  lot  of  hard 
problems  but  I  made  it  through  those.    When  I  had  new  problems  I  needed  a 
lot  of  help,  so  I  called  on  Miss  V.     I  call  her  Miss  V.  for  short.  I 
thought  it  was  fun  because  it  was  on  a  computer.    You  didn't  have  to 
write  it  on  paper.     I  almost  toped  out  at  the  e^d.    My  average  was  78, 
Sometimes  I  got  mad  because  I  got  a  lot  of  problems  wrong.     I  learned  a 
lot  in  computers.    The  computer  taught  me  a  lot  about  math.     I  was 
lucky  because  math  is  my  favorite  subject.     I  liked  my  teacher  Miss  V. 
she  was  very  nice.     I  am  going  to  miss  that  computer  room  very  much. 

*  *  *  * 

To  whom  it  may  concern: 

I  thin  you  should  leave  the  C.A.I,  program  running. 

I  had  a  lot  of  help  from  the  computers  in  language  and  in  reading.  They 
teach  you  a  better  vocabulary  than  you  already  have.     Because  it  sure 
improved  my  vocabulary  alot  better  than  it  used  to  be. 

So  I  think  C.A.I,  should  proceed  as  it  does  now. 

*  *  *  * 

How  I  feel  about  the  C.A.I. 

The  C.A.I,  is  real  boring  but  I  guess  it  would  be  better  if  I  had  reading 
or  language,  but  I'm  spuclc  with  math. 

It  also  seems  to  waste  time  in  my  regular  class  &  I've  never  gotten  100% 
in  my  course. 

The  teachers,  Mrs.  V.  &  Renee  are  very  smart  &  its  fun  to  talk  with  them. 
I  think  that  even  math  is  a  good  course*  for  me  because,  I  want  to  be  a 
doctor  or  a  Dentist  and  you  need  to  know  alot  of  math  to  do  these  things. 
(I'm  not  very  good  in  math.) 

*  *  *  * 

I  like  to  do  the  computers  becase  I  lern  a  lot. 

I  think  that  evre  elemetry  school  shold  have  CAI  what  I  mean  is  evere 
school  in  the  world  shold  have  CAI. 

it   ^  it  it 
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I  am  proud  of  myself  on  the  computers. 

It  is  a  neat  experience  to  work  on  the  computers. 

I  think  I  have  gotcen  some  good  scores. 

The  computer  teacher  is  nice  and  when  ever  I  need  help  she's  there. 

*  *  *  * 

When  I  first  started  computer  I  really  liked  it.    as  I  got  older  I  didn't 
like  it  a  as  much.     Now  I'm  in  fifth  grade  and  I  dont  like  computer.  It 
is  a  good  program  but  It  doesn't  take  my  fancy.     I  don't  like  it  because 
it  gets  me  uptight  and  it  makes  me  miserable.     It  makes  me  uptight  and 
miserable  because  it  is  a  computer,  not  a  person,  and  computers  are 
taking  over  the  world!    The'  End 

*  *  *  * 

I  liked  the  CAI  room  last  year  because  I  liked  the  work  that  I  did  there. 
It's  fun  because  I  don't  have  to  write  with  a, pencil.    We  could  only  type 
with  are  hand  you  can  not  write  with  a  pencil  and  they  help  us  learn  the 
words  like  language  or  reading  or  math  we  learned  a  lot' of  things    In  the 
CAI  room  that's  why  I'm  sorry  we  dont  go  to  the  CAI  room  any  more  and 
never  again  my  my  whole  life.     The  end. 

*  *  *  * 

1.  I  like  the  computer  room  because  it  s  nice. 

2.  I  might  become  a  computer  teacher  some  day. 

3.  The  computers  are  nice  to  work  on. 

It  must  be  emphasized  again  that  the  selection  of  student  letters 
was  intentionally  biased  toward  negative  comments  in  order  to  provide 
information  about  problem  areas.    A  caution  is  in  order,  however.  Most 
of  the  **boredom"  responses  were  a  direct  result  of  the  research  design  by 
which  some  students  received  nothing  but  mathematics  CAI  over  periods  of 
three  or  four  years  while  their  classmates  received  both  reading  and 
language  CAI  or  a  combination  of  all  three  CAI  curriculums.     In  general, 
students  en joyed othe  CAI  labs,  behaved  well  and  attended  to  task. 


Er|c  ^3 
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Teachers'  Opinions 

Each  year  teachers  with  CAI  students  were  interviewed  during  the 
final  week  of  school.    This  section  summarizes  the  responses  of  these 
teachers  in  three  areas:     their  evaluation  of  the  CAI  program,  their 
ideas  about  how  CAI  is  helpful  to  students,  and  their  feelings  about  t^he 
transition  of  CAI  from  a  federally-funded  project  to  a  ^chool  program. 

Program  Evaluations 

In  Chapter  II  it  was  mentioned  that  initial  acceptance  of  the  CAI 
program  was  less  than  whole-hearted  due,  at  least  in  part,  to  start-up 
problems  with  the  equipment  and  interruptions  to  the  classroom  organization 
in  midyear.    During  years  2-4  of  the  study,  teacher  attitudes  about  CAI 
continued  to  be  monitored.    At.  the  end  of  each  year  teachers  of  CAI 
students  were  interviewed  and  asked  for  their  ratings  on  the  following 
evaluative  scales: 

Overall,  was  the  CAI  program: 

well  run  1. . .2. . .3. . .4. . .5  chaotic 

harmful  to  pupils  5. . .4. . .3. . .2. . .1  helpful 

disruptive  non-disruptive 
to  class  5. . .4. . .3. . .2. . .1      to  class 

worthwhile  1. . . 2. . . 3. . .4. . . 5  worthless 

equipment  equipment 
undependable  5...4...3...2...1  dependable 

successful  1...2..«3...4...5  unsuccessful 

Are  you  glad  or  sorry  to  have  had  CAI  ia  your  school  this  year? 
  '    glad  "  sorry    mixed  feelings 


0\i 
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Mean  ratings  by,  teachers  in  years  2  through  A  on  the  program  evaluation 
scales  are  given  in  Table  1.     Data  from  year  A  are  divided  into  2  sections: 
data  from  Schools  1-3  and  data  from  School  A.    Since  School  A  had  a 
difficult  year  internally  in  1980  and  also  had  a  new  CAl  coordinator, 
results  from  School  A  are  perhaps  less  representative  of  CAI  than  of 
upheaval,  in  the  school.    Having  reported  the  data  from  School  4,  we  will 
not  consider  it  further  except  to  say,  even  here,  the  responses  are  more 
positive  than  negative.    In  general  the  ratings  improve  over  time. 

The  positive  evaluations  in  Schools  1-3  in  the  final  year  are  in 
contrast  with  the  previously  reported  results  for  year  1  of  the  study. 
At  the  end  of  the  first  year,  there  were  serious  reservations  about  CAI 
and  the  way  it  operated.    Now,  at  the  end  of  the  study,  it  apears  that 
these  negative  attitudes  have  largely  been  overcome,  to  produce  what  can 
be  reasonably  described  as  a  vote  of  confidence  from  the  teachers.  All 
teachers  in  Schools  1-3  in  the  final  year  of  the  study  reported  being 
glad  to  have  had  CAI.     (See  Table  2.) 

Scales  regarding  specific  features  of  the  CAI  program  give  an 
indication  of  strengths  and  weaknesses.    The  most  negative  evaluations 
were  of  dependability  of  the  equipment.    Teachers  occasionally  reported 
breakdowns 'which  caused  disruption  because  of  the  necessary  rescheduling 
and  students'  frustration.    Some  concern  was  expressed  about  disruption 
to  class,  although  the  large  majority  of  teachers  still  rated  the  experi- 
ence as  nondisruptive.    The  program  got  very  high  marks  for  being  well 
run,  helpful,  worthwhile,  and  successful. 

Each  year  teachers  were  given  a  chance  to  make  open-ended  suggestions 
by  explaining  what  they  would  plan  if  they  controlled  the  program.  Most 
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Table  1 

Mean  Ratings  by  Teachers  in  Years  2-4 
on  Program  Evaluation  Scales 


Year  2 
(All  Schools) 

Year  3 
(All  Schools) 

Year 

(Schools  1-3) 

4 

(School 

Well  run: 

1.3 

1.2 

1.0 

2.6 

Helpful: 

1.6 

1.2 

1.0 

2.2 

Non-disruptive: 

1.5 

1.7 

.1.5 

2.5 

Worthwhile: 

1.6 

1.3 

1.0 

2.6 

Equipment  dependable: 

1.9 

2.2 

1.6 

2.0 

Successful 

1.5 

1.4 

1.0 

2.6 

Table  2 

Percentage  of  Teachers  Reporting  Being 
Glad  or  Sorry  to  have  Had  CAI  in  Years  2-4 

Year  2  Year  3  Year  4' 

(All  Schools)     (All  Schools)     (School  1-3)  (School  4) 

Glad:  88%      ^  91%  100%  58% 

Mixed  Feelings:  12%       .  9%  —  28% 


Sorry; 
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14% 
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of  the  changes  proposed  had  to  do  with  the  conditions  of  the  experiment 


rather  than  with  CAI  itself.    Most  teachers  appeared  to  understand  the 


necessity  of  abiding  by  the  experimental  conditions  but  still  chafed 


under  the  restraints.    The  most  frequent  suggestions  were  to  assign 
,  students  to  pi^grams  according  to  need.    The  most  frequent  suggesition  of 


this  type  was  to  use  CAI  for  remediation  when  needed.    Another  frequent 
suggestion  was  to  use  CAI  for  enrichment  and  stimulation  of  more  advanced 
students.    Another  frequently  suggested  change  called  for  coordinating 
"CAI  lessons  with  the  curriculum.    A  specific  desire  was  the  ability  to 
use  CAI  for  reinforcement — e.g.,  to  give  practice  itit  a  skill  after  a 
lesson  in  that  skill.    Several  teachers  mentioned  changes  in  the  scheduling. 
However  there  were  no  specific  changes  that  were  mentioned  frequently 
enough  to  Justify  a  recommendation  regarding  scheduling.    Answers  vary 
with  teachers*  styles  and  experience.  - 

CAI  Effects  on  Students 

Teachers  were  asked  to  discuss  examples  of  students  who  profited 
especially  from  CAI  and  students  for  whom  it  was  a  waste  of  time. 
Relatively  few  instances  of  waste  of  time  were  mentioned.     The  ones  that 
were  mentioned  involved  students  who  could  not  concentrate,  did  not 
cooperate,  or  had  learning  problems  that  mader  learning  difficult  in  all 
Bettings.    Few  negative  cases  were  cited.     One  negative  result  that 
sometimes  occurred  was  the  assignment  of  non-English  speaking  students  to 
CAI  programs  in  reading  and  language,  causing  increased  frustration.  c. 
This  resulted  from  adherence  to  random  assignment  in  the  Resign  of  the 


experiment. 


ERIC 


The  teachers  suggested  a  number  of  hypotheses  about  circumstances 
in  which  CAI  might  have  especially  beneficial  effects.    Two  groups  of 
students  were  mentioned  repeatedly.    The  first  was  students  who  have 
limited  English  speaking  ability.     When  assigned  to  CAI  mathematics  (as 
contrasted  with  reading  and  language),  these  students  had  a  chance  to  go 
ahead  with  minimal  frustration  from  language  problems.    The  other  frequently 
mentioned  groups  were,  above-average  and  gifted  students,  who  get  an 
opportunity  to  move  ahead  at  their  own  pace.     Several  teachers  reported 
being  asked  to  explain  concepts  encountered  in  CAI  but  not  covered  in 
class.    Often  with  a  little  bit  of  instruction  from  the  teacher  and 
coordinators,  the  student  was  able  to  progress. 

A  number  of  ter.chers  mentioned  motivational  values,  which  especially 

helped  some  students.    Here  are  some  examples: 

o    CAI  was  credited  in  building  self-confidence  in  a  number  of 

students.    The'  feedback  gives  them  confidence  to  try  tasks  they 
otherwise  shy  away  from. 

o    Concrete  knowledge  of  results  encour^^es  persistence. 

o    Awards  and  competition  provide  reinforcement  that  was  highly 
motivating  to  students. 

o    The  break  from  routine  is  motivating  for  some  students. 

Teachers  often  mentioned  features  of  the  process  of. CAI  itself  that 

are  advantageous  for  some  students.    Among  these  were  the  following: 

o    The  machine  requires  concentration  and  discipline.     It  provides 
structure.    This  is  particularly  useful  for  students  who  need 
training  in  self-discipline. 

o    The  machine  is  benign.     It  does  not  require  verbalization, 

putting  ideas  on  paper  or  interacting  with  peers.    Therefore  CAI 
is  an  effective  method  for  students  who  have  problems  in  these 
areas. 
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There  were  a  number  of  other  ideas  whlch'were  mentioned  by  only  one 

or  two  teachers  and  which  provide  useful  hypotheses. 

o    Many  students  leave  the  area  for  weeks  at  a  time  and  then  return. 
CAI  picks  them  up  individually  where  they  left  off. 

o    The  computer  does  not  "label"  low  achievers  as  do  reading  books 
and  reading  groups. 

o    The  short  problems  and  immediate  feedback  are  good  fo^  students 
with  short  attention  spans.  -  '  e- 

The  Future  of  the  CAI  Program 

During  the  final  year  of  field  operations  for  the  ETS-LAUSD  Computer- 
Assisted  Instruction  Project,  teachers  were  asked  the  following  questions: 

If  there  were  no  coordinator  next  year  would  you.  f&el 
comfortable  bringing  your  class  to  the  CAI  lab  on  your  own? 

••        Would  you  feel  the  need  for  in-service  training  on  the  com- 
puter?   The  terminals?    The  CAI ""curriculums? 

If  you  had  your  choice,  how  would  you  use  (or  not  use)  the 
CAI  lab  next  year? 

What  problems  do  you  foresee  for  the  use  of  CAl  in  future 
years? 

Almost  two-third^  of.  the  teachers  responded  that-  they  would  be  comfortable 

n 

on  their  own  in  the  CAI  lab.    Some  expressed  a  preference  f or  mana'ging 
CAI  themselves,  although  many  more  were  concerned  about  the  lack  of  a 
coordinator.    Schools  with  small  CAI  labs — with  16  terminals  instead  of 
32— were- especially  bothered  by  the  lack  of  funds  for  a  coordinator.  "I 
don't  know  quite  how  it's  going  to  work — bringing  my  whole  class  to  the 
lab  with  only  enough  terminals  for  half.    I'm  not  sure  how  to  handle  the 
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children  who  can't  be  on  the  terminals.    But  we'll  work  it  out."  Only 
a  few  teachers  were  reluctant  to  be  in  the  CAI  lab  on  their  own,  and 
their  perceptions  were  probably  best  expressed  by  the  teacher  who  said, 
"No  way!     The  technical  problems  would  be  terrifying."    Only  two  of  the 
41  teachers  balked  at  in-service  training  on  the  computer  program:  one 
because  she  felt  she  was  completely  nonmechanical  and  another  because  she 
was  already  overworked.    Thirty-nine  teachers  responded  positively  to  the 
need  for  in-service  training.    Many  felt  they  knew  a  great  deal  already 
but  could  use  additional  help  on  the  more  technical  aspects  of  the  CAI 
operation. 

Given  their  choice,  37  of  the  41  teachers  interviewed  would  elect  to 
take  their  pupils  to  the  CAI  lab  three  to  five  times  a  week.    One  teacher 
would  choose  not  to  go  to  the  lab  and  two  others  would  send  only  selected 
students.    One    teacher  had  not  made  up  her  mind.    Eight  teachers  would 
prefer  to  go  to  CAI  only  three  days  a  week,  while  29  would  elect  to  go  on 
a  daily  basis.     Of  those  who  specifically  mentioned  whether  they  would 
prefer  to  have  one  CAI  session  or  two,  more  than  60%  chose  to  have  their 
students  remain  in  the  lab  for  two  sessions  of  CAI.    Mathematics  was  the 
CAI  curriculum  mentioned  specifically  most  often,  although  many  teachers 
mentioned  selecting  the  CAI  curriculum  on  the  basis  of  need.     (None  of 
them  recommended  continuation  of  the  process  of  random  assignment  of 
students  to  curriculums! ) 

,       The  lack  of  money  to  run  the  CAI  lab,  the  loss  of  a  full-time 
coordinator,  and  the  maintenance  of  the  equipment  topped  the  list  of 


/ 

.    /  /' 

problems  for  CAI  in  making  the  transition . from  a  federally-funded  project 

to  a  school  program.    More  than  half  of  the  teachers  mentioned  at 

/ 

least  one  of  those  problems.    Staff  training  or  in-service  was  mentioned 
as  a  problem  by  six  teachers.    Six  teachers  mentioned  there  would  be  no 
problem,  and  several  mentioned  computers  as  the  tool  of  the  future, 
receiving  more  and  more  use  in  the  ed/icational  community.    Labk  of 
control  in  the  CAI  lab  was  reported  as  a  possible  problem  by/ three 
teachers  and  scheduling  by  four  others* 

Despite  the  problems  mentionecy  realistically  by  teache/rs,  attitudes 
toward  the  future  of  CAI  in  the  four  schools  were  generally  optimistic. 

/ 

Teachers  knew  that  the  transition/would  not  be  accomplished  without 
problems,  but  they  had  little  doubt  of  their  own  ability  ^to  cope. 

/ 

The  Principals'  Perspective 

The  principal  of  each  CAI  school  was  interviewed^ at  the  end  of  the 
project,  and  all  four  were  unreservedly  enthusiastic  about  the 
success  of  the  program  through  the  first  four  years.     In  addition  to  the 
students'  academic  gains,  the  principals  cited  other  valuable  educational 
outcomes  such  as  providing  students  with  the  experience  of  success  and 
teaching  them  the  discipline  of  staying  at  a  task.    The  principals  Valued 
the  fact  that  the  program  uses  positive  reinforcement  and  felt  that  the 
work  was  interesting  and  challenging  for  their  students.    Three  of  the 
principals  made  particular  mention  of  the  very  positive  attitude  of 
parents  toward  the  CAI  program  and  credited  the  program  with  improving 
the  relationship  between  the  school  and  the  community. 
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All  principals  expected  to 
during^the  1980-81  school  year, 
transition  year  is  the  topic  of 


operate  the  CAT  labs  in  their  schools 
What  actually  happened  during  that 
Chapter  IV. 


Summary 


In  Chapter  III  we  have  reported  on  evaluations  of  computer-assisted 
instruction  from  several  perspectives.     Coordinators  of  CAI  labs  wrote  of 
their  successes  and  failures.     Students  wrote  of  their  pleasure  and 
frustration.    Teachers  evaluated  the  CAI  program,  discussed  its  effects 
on  .students  and  contemplated  its  future.    Attitudes  of  students  and 
school  personnel  were  overwhelmingly  positive  but  problems  were  reported. 
Principals  expressed  concern  about  the  transition  of  CAI  from  a  federally 
funded  project  to  a  school  program  financed  by  the  Los  Angeles  Unified 
School  District.    A  report  on  that  transition  is  the  subject  of  Chapter  IV. 


Chapter  IV 
TRANSITION* 


Of  concern  to  teachers f  coordinators  and  principals  was  the 
transition  of  the  CAI  program  from  a  federally  funded  project  \o  a 
program  run  by  the  schools.     This  chapter  describes  the  transition 
during  the  final  year  of  the  project  and  the  first  year  without 

■s 

project  funding.     There  are  two  sections:    a  diary  of  the  activities  of 
the  transition  and  a  report  on  interviews  with  school  personnel  at  the 
end  of  the  year  of  transition. 

The  Diary  of  Transition 

October,  1979 

Planning  for  the  transition  from  an  externally  sponsored  project 
to  a  school  district  program  administered  by  the  individual  schools 
began  in  October  of  1979  with  a  meeting  called  by  Dr.  Warren  Juhnke, 
Deputy  Area  Administrator  of  Area  4  and  director  of  the  project  for  the 
Los  Angeies^tJnified  School  District.     Present  at  th^t  meeting  were 
the  principals  and  CAI  coordinators  of  the  four  CAI  schools;  Ms. 
Roberta  Woodson,  coordinator  for  three  years  at  School  4;  Ms.  Judy 
Newman,  substitute  coordinator  at  all  four  schools,  interviewer,  classroom 
observer,  and  testing  coordinator;  Dr.  Marjorie  Ragosta,  ETS  Project 
"Director;  and  several  Area  4  administrators,  with  responsibilities  in  the 


*The  major  portion  of  this  chapter  was  written  by  Puff  Rice. 
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areas  of  learning  disabilities,  curriculum  coordination,  and  general 
administration.    Throughout  the  year  of  planning,  this  remained  the  core 
group,  sometimes  augmented  by  other  specialists  and  administrators. 

At  this  first  meeting  the  two  immediate  responsibilities  of  each 
school  were  established — to  develop  an  individual  plan  for  a  CAI  program 
fot  1980-81,  and  to  begin  at  once  to  search  for  funding  sources. 
None  of  the  participants  expressed  the  slightest  doubt  about  wanting 
or  needing  CAI,  nor  did  they  express  any  hesitation  about  assuming 
full  responsibility  for  running  the  program.     The  question  of  funding, 
however,  especially  in  light  of  the  scarcity  of  funds  since  the  passage 
of  Proposition  13,  was  seen  as  a  serious  problem.     It  was  suggested 
that  the  Management  Information  Division  of  LAUSD  might  support  the 
maintenance  of  the  equipment  and  possibly  even  the  rental  of  curriculums. 
Funds  designated  for  instructional  materials  were  also  suggested  as  a 
source  for  curriculum  rental  money.     It  was  felt  that  money  for  personnel 
to  run  the  CAI  labs  would  be  the  most  difficult  to  obtain.  Various 
possibilities  were  identified  for  exploration:    Title  I  funds,  bilingual 
education  funds,  Master  Plan  funds,  ESEA  funds  (including  funds  designated 
for  racially  isolated  minority  schools — RIM),  special  federal  or  state 
allocations.  Area  4's  own  money,  a  program  plan  with  a  budget  submitted 
directly' to  the  school  board,  and  the  "Adopt-a-School"  program. 

.The  ^itiCude  of  the  participants  at  this  meeting  was  very  positive, 
even  while  recognizing  the  reality  of  the  funding  problem.     The  group 
agreed  to  meet  again  In  December  to  check  on  progress* 
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December.  1979 

In  addition  to  the  core  group,  several  Area  4  and  district  level 
administrators  and  consultants  attended  the  December  meeting,  representing 
Master  Plan,  special  education,  curriculum  services,  learning  disabilities, 
and  the  Management  Information  Division.    Mrs.  Eugenia  Scott,  Area  4 
Superintendant,  was  unable  to  attend.     Dr.  Ragosta,  who  was  also  unable 
to  attend,  sent  another  member  of  the  ETS  project  staff.    The  purpose 
of  this  meeting  was  to  bring  together  and  discuss  the  individual  school 
plans. 

Schools  1  and  2  had  prepared  a,  joint  plan,  since  they  shared  one 
computer.    Their  plan  called  for  serving  the  entire  population  of  each 
school,  including  Special  Day  Classes.    An  adult  ESL  class  would  also 
be  served  at  School  2.     Because  the  CAI  lab  at  each  school  was  designed 
to  accommodate  only  half  a  class,  a  coordinator  would  be  needed  at 
each  school,  so  that  children  would  never  be  left  without  a  certificated 
teacher  in  charge.     School  4's  plan  also  called  for  serving  the  entire 
school,  including  Special  Day  Classes.     Their  plan  suggested  one 
coordinator  to  serve  all  four  schools,  with  each  school  paying  25%  of 
the  coordinator's  salary.    School  3  also  suggested  a  traveling  coordinator 
but  added  the  idea  of  one  regular  teacher  at  each  school  having  responsi- 
bility for  that  school's  CAI  program.     School  3  planned  to  involve  all 
regular  classes  in  the  CAI  program  with  the  hope  of  expanding  to  include 

special  education  students  and  an  ^dult  education  class.  ^\ 

/ 

On  the  subject  of  funding.  Dr.  Juhnke  estimated  a  total  cost  for 
the  four  schools  of  $200,000.     Schools  1  and  2  presented  a  joint  budget  / 
of  $89,033.     They  had  no  idea  where  that  money  might  come  from^.     School  3 
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submltted  an  optimum  budget  of  $51,366  (with  a  full-time  coordinator) 
and  a  marginal  budget  of  $30,588  (with  25%  of  a  coordinator).'  \They 
expected  to  be  able  to  support  two  teacher  aides  from  their  Titl^  I 
funds  but  would  need  outside  support  for  the  rest.     School  4,  altWugh 
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they  had  not  prepared  a  formal  budget,  estimated  that  they  could  pi\^yide 
two  teacher  aides  from  local  school  funds  and  25%  of  a  coordinator  ^rom 
their  Title  I  funds.   ;£ven  if  the  costs  of  hardware  maintenance  and  ^ 
curriculum  rental  were  to  be  provided  by  the  Management  Information 
Division,  as  the  administrators  thought  was  reasonable  to  expect, 
funding  for  the  bulk  of  the  personnel  costs  was  still  a  probljem  of 
immense  proportions.    All  the  schools  firmly  maintained  that  a  coordinator 
was  essential. 

Facing  the  possibility  of  insuf f ici^njt  funding  for  the  kinds  of 
plans  outlined  by  the  four  schools,  participants  in  the  meeting  considered 
a  number  of  other  imaginative  approaches.    It  was  suggested  that  separate 
funds  might  be  obtained  for  adult  education  programs,  although  such  money 
probably  could  not  be  applied  to  the  regular  school  program.    The  principal 
of  School  3  indicated  that  he  preferred  "parent  education,"  which  is 
under  the  control  of  the  local  school,  whereas  "adult  education"  is  not. 
He  felt  that  a  night  program  at  School  3  would  pose  a  serious  security 
problem.    A  parent  education  program  was  also  suggested  for  Schools  1  and 
2,  combined  with  a  half-day  program  of  CAI  for  the  children  at  each 
school.     It  was  felt  that  the  locals  parents  of  School  1  either  did  not 
need  or  would  not  support  such  a  program;  the  parents  of  children  who  are 
bussed  from  the  other  schools  in  the  integration  triad  to  which  School  1 
belonged  lived  too  far  away  to  participate. 
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A  question  was  raised  about  the  advisability  of  suggesting  that 
the  A-16  computers  be  "traded  in"  .for  A-l7's,  for  which  more  varied' 
curriculums  are  available.  The  consensus,  especially  among  the  core 
group,  was  that  the  schools  needed  to  convince  the  school  board  that 
the  equipment  and  the  program  they  already  had  were  good  and  should  be 
maintained* 

A  proposal  for  Title  IVc  funding  was  suggested,  but  the  administrators 
present  felt  that  the  CAI  program  could  not  qualify,  since  it  would  be 
considered  an  ongoing  program  rather  than  a  new  one.    It  was  also 
pointed  out  that  the  cotapetition  for  such  money  was  fierce,  and  the 
grants  were  small. 

Since  School  1  and  School  2  each  had  only  enough  terminals  for 
half  a  class,  it  was  suggested  that  the  terminals  all  be  housed  in  one 
school  and  the  children  be  bussed  from  the  other  school.    Although  this 
would  eliminate  the  necetisity  of  having  a  certificated  teacher  in 
charge  of  each  half  of  the  class,  the  cost  (in  both  time  and  money) 
and  the  administrative  cumbersomeness  of  bussing  more  than  outweighed 
the  advantages. 

One  of  the  consultants  suggested  relo'zating  the  terminals  in 
regular  classrooms,  two  or  three  to  a  room,  so  that  classroom  teachers 
could  use  them  like  resource  centers.    This  would  completely  eliminate 
the  costs  of  coordinators  and  aides.    It  was  felt  that  this  would  alter 
the  nature  of  the  program  too  drastically  and  would  very  likely  result 
in  greatly  reduced  benefit  to  children. 

Summing  up  the  meeting.  Dr.  Juhnke  directed  each  CAI  school  to 
decide  what  program  would  be  best  for  them,  even  if  their  program  was 
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different  from  everyone  else's.    He  emphasized  that  "No  Program"  was 
ndt  an  option;  although  the  funding  agency  had  agreed  to  turn  over  the 
equipment  to  the  schools  at  the  end  of  the  research  project,  it  had 
done  so  with  the  understanding  that  the  equipment  would  be  used. 

The  attitude  of  grave  concern  for  the  future  of  the  program  that 
was  expressed  at  the  meeting  was  consonant  with  the  feelings  of  the 
coordinators  when  they  were  interviewed  several  days  prior  to  the 
meeting.     They  felt  that  no  progress  was  being  made  in  funding  and  that 
unless  outside  money  were  found  the  program  could  not  function  at  all 
in  1980-81,     Even  the  prospect  of  money  from  the  Management  Information 
Division  for  maintenance  and  curriculum  rental  did  nothing  to  alleviate 
their  fears.    They  all  felt  that  the  program  would  not  be  use^d  without 
a  coordinator.    One  coordinator  reported  that  she  had  tried  to. train 
five  primary  teachers  to  handle  a  class  in  the  lab.    Four  gave  up  after 
a  short  while;  one  continued  to  try  but  was  ineffective.    The  coordinators 
were  considerably  more  pessimistic  than  their -principals  and  the  Area  4 
administrators  about  the  fate  of  the  CAI  program  in  1980-81* 

January,  1980 

A  composite  propbsal  for  a  CAI  program  for  1980-81,  containing 
the  final  individual  plans  and  budgets  of  the  four  schools  was  submitted 
to  the  Area  A  Superintendent  hy  0^r.  Juhnke  in  January  of  1980. 

The  planned  program  for  School  1  called  for  all  pupils  in  grades 
1-^3  to  receive  CAI  for  20  minutes  each  week.    All  fourth  graders  would 
receive  SO  minutes  of  CAI  each  week.     Individualized  instruction  would 
be  provided  for  Special  Day  Classes,  ESL  children,  and  mains treamed 


children.     (There  were  no  fifth  or  sirth  grades  at  School  1  because  of 
the  integration  triad  arrangement.)    The  cost  for  the  program  at  School  1 
was  $31,727. 

School  2  proposed  providing  CAI  for  all  students  in  grades  1-6  for 
a  minimum  of  50  minutes  per  week.     Children  in  Special  Day  Classes  and 
ESL  children  would  receive  individualized  instruction.     School  2  planned 
to  continue  to  offer  ESL  parent  education  using  CAI.     The  budget  for 
School  2,  including  hardware  and  software  costs  for  School  1,  was  $53,754. 
(School  1  used  the  computer  housed  in  School  2«) 

School  3  planned  to  provide  CAl  for  all  pupils  in  grades  1-6, 
with  the  amount  of  CAI  time  per  week  varying  according  to  need.  Individual 
ized  instruction  would  be  provided  for  pupils  with  exceptional  needs.  A 
community  adult  education  plan  would  also  be  offered.     School  3  proposed 
a  minimum' budget  of  $29,340  and  an  optimum  budget  of  $51,013. 

School  4  proposed  a  CAI  program  for  all  pupils  in  grades  1-6, 
for  varying  amounts  of  time  according  to  need.     The  minimum  CAI  time 
for  any  student  would  be  ten  minutes  daily  for  one  semester.     School  4 
also  proposed  individualized  instruction  for  pupils  with  exceptional 
needs  and  a  community  adult  education  program  in  basic  skills  and  ESL. 
A  unique  feature  of  School  4's  plan  was  an  enrichment  program  for  state 
identified  gifted  students.    School  4's  budget  was  $29,449. 

It  should  be  noted  that  all  of  the  final  budgets  were  slightly 
lower  than  the  original  estimates  discussed  at  the  December  meeting. 
The  optimum  total  budget  ($165,943)  was  $34,000  lower  and, the  minimum 
total  budget  ($144,270)  was  $56,000  lower  than  the  estimated  total  of 
$200,000.    Possible  sources  of  funds  were  suggested  in  the  proposal. 
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including  Title  I  (for  Schools  3  and  4)^  integration  funds  (for  School  1), 
the  Management  Information  Division  (for  hardware  maintenance  and 
curriculum  rental )y  ESEA  funds.  Master  Plan  for  Special  Education  funds, 
a  grant  from  the  Bureau  of  Education  for  the  Handicapped,  bilingual 
education  money,  state  identified  gift  funda,  and  private  industry. 

The  tone  of  the  proposal  was  quite  strong  and  positive,  citing  the 
record  of  success  already  established  by  CAI  and  the  monetary  value 
of  the  equipment  soon  to  be  transferred  into  the  school  district's 
possession*     The  proposal  pointed  out  the  strong  foundationiof  trained 
staff  and  of  community  and  teacher  support  that  had  been  developed  over 
the  first  three  years  of  the  project.    The  many  ways  in  which  CAI  saved 
classroom  teacher  time  while  providing  documented  educational  benefits 
to  students  were  enumerated.    The  proposal  closed  by  stating  the  willing* 
ness  of  all  four  schools  to  work  with  the  Evaluation  and  Research 
Branch  of  LAUSD  to  build  evaluation  components  into  their  programs. 

May,  1980  ^ 

At  a  meeting  of  CAI  principals  and  coordinators  in  May,  Dr.  Juhnke 
summarized  the  progress  of  the  composite  CAI  proposal  since  January. 
The  proposal  was  submitted  to  the  deputy  superintendent  of  the  district 
in  January.     The  CAI  principals,  one  of  the  coordinators,  the  Area  4 
superintendent,  and  Dr.  Juhnke  met  with  him  in  February  to  discuss  it. 
The  deputy  superintendent  strongly  supported  the  proposal  and  appointed  a 
committee  to  investigate  possible  funding  sources.    That  committee  met  in 
March.    The  results  of  its  deliberations  were  negative:    any  Title  I 
monies  applied  to  CAI  programs  would  have  to  be  drawn  from  the  local 
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school's  regular  Title  I  budget;  although  schools  that  received  School 
Improvement  Program  funds  could  elect  to  use  those  funds  for  CAI,  no 
additional  SIP  funds  would  be  forthcoming;  there  would  be  no  involvement 
with  adult  education  programs. 

The  only  positive  news  about  funding  was ° that  the  Management 
Information  Division  had  received  a  lower  bid  than  anticipated  for 
hardware  maintenance  and  expected  to  be  able  to  cover  that  expense  as 
well  as  most  if  not  all  of  the  curriculum  rental  costs.    By  way  of 
underscoring  the  severity  of  the  school  district's  financial  plight, 
participants  in  the  meeting  were  informed  that  the  district's  expenditures 
had  exceeded  its  income  in  1979-80  by  $20,000,000,  and  that  $60,000,000 
had  been  cut  from  1980-81  program  funds' in  the  preliminary  budget. 
Although  he  remained  stubbornly  optimistic.  Dr.  Juhnke  realistically 
advised  the  coordinators  to  make  alternate  emplo3rment  plans  for  themselves 
for  the  coming  year. 

The  possibility  of  applying  for  a  Bank  of  America  grant  was  dis- 
cussed.    It  was  felt  thkt  individual  schools  stood  a  better  chance  of 
receiving  such  grants  than  would  the  four  schools  applying  together.  The 
grants  would  not  be  large  enough  to  cover  the  cost  of  a  coordinator, 
-however.    The  Bureau  of  Education  for  the  Handicapped  was  also  considered, 
and  preliminary  investigation  indicated  that  they  would  consider  a  small 
proposal  for  a  study  of  the  use  of  CAI  for  the  handicapped,  but  such  a 
proposal  could  not  be  funded  by  September.     The  individual  schools 
were  instructed  to  look  at  their  proposals  again,  eliminating  the 
hardware  and  software  costs,  and  to  see  how  close  they  could  come  to 
meeting  the  rest  of  the  costs  through  their  local  budgets. 


Dr.  Juhnke  reaffirmed  his  belief  that  the  deputy  superintendent  was 
committed  to  the  program  and  that  LAUSD  would  not  let  the  equipment 
sit  idle  and  the  coordinators'  expertise  go  to  waste.     The  Area  4 
superintendent,  who  was  able  to  come  to  the  last  part  of  the  meeting, 
agreed  that  the  financial  picture  was  dismal  but  stated  that  she  had 
faith  that  a  way  would.be  found.     The  coordinators  were  disappointed 
and  downcast.    The  principals — particularly  of  Schools  3  and  4— 
continued  to  wrestle  with  the  problem,  searching  for  some  way  to  carve 
out  enough  money  to  keep  the  program  alive.     It  was  generally  acknowledged 
that  without  a  coordinator  the  program  simply  could  not  serve  as  many 
children  effectively  as  it  had  before. 

June,  1980 

The  principal  of  each  CAI  school  was  interviewed  at  the  end  of  the 
school  year  to  discuss  their  plans  for  1980-81.     Looking  ahead  to  the 
first  year  of  operation  as  a  school-^run  program  the  principals  expressed 
several  concerns  that  were  common  across  all  four  schools.    Foremost  was 
the  lack  of  funds  for  coordinators.    At  this  point  the  principals  had 
abandoned  hope  of  any  funds  other  than  those  to  be  provided  by  the 
Management  Information  Division.    All  the  principals  felt  that  the 
coordinator  was  the  key  factor  in  making  the  program  run  well  J  and  they 
expressed  keen  regret  that  without  a  coordinator  the  program  probably 
would  not  be  able  to  serve  all  of  the  children  that  it  had  the  potential 
to  serve.    Without  a  coordinator,  individual  teachers  must  learn  much 
more  about  managing  the  program  than  they  had  to  know  before,  and  that 
meant  inservice  training,  another  problem  of  time  and  money. 


A  second  common  problem  was  the  general  uncertainty  about  school 
enrollments,  which  left  such  matters  as  class  size,  number  of,  classes, 
assignment  of  faculty,  and  even  budgets  to  be  settled  some  time  after 
school  opened  in  the  fall.     Although  this"  was  an  uncertainty  which 
administrators  always  had  to  live  with  in  Los  Angeles,  the  situation 
was  further  complicated  by  the  district-wide  integration  program  that 
was  in  the  process  of  being  challenged  and  redesigned.    School  1  and 
School  3  were  each  involved  in  cooperative  .integration -plans  with 
schools  in  other  locations. 

The  Title  I  schools  (3  and  4)  had  some  similar  features  that 
influenced  their  plans  for  running  their  CAI. programs  in  1980-81. 
Both  schools  were  large.     Both  had  enough  terminals  in  the  CAI  lab 
to  accommodate  an  entire  class,  so  that  the  regular  classroom  teacher 
could  accompany  students  to  the  lab,  and  all  students  in  the.Qlass 
could  work  on  the  terminals  at  the  same  time.    The  Title  I  budgets 
provided  some  funding  flexibility  that  the  non-Title  I  schools  did  not 
have.     Two  teacher  aides  for  the  CAI  lah  were  wri/ten  into  the  Title  I 
budget  for  each  school.     At  School  3  the  same  aides  wRo  worked  in  the 
lab  in  1979-80  expected  to  return  in  1980-81.     School  3's  Title  I 
budget  also  included  funds  for  a  "professional  expert"  to  spend  some 
time  training  teachers  to  use  the  system.     School  3  expected  to  hire 
as  a  "professional  expert"  their  former  CAI  coordinator,  who  planned 
to  take  a  year's  leave  in  1980-81.     At  School  4  the  Title  I  budget 
included  money  for  25%  of  a  coordinator,  which  was  a  possibility  being 


considered  at  the  time  budgets  were  being  prepared.    Although  the 
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school  district  had  abandoned  that  plan,  the  money  was  still  in  the  c 
budget,  and  School  4  had  npt  yet  decided  how  to  make  the  best  use  of  it. 

School  3  had  made  definite  plans  for  1980-81,  involving  the  groups 
that  the  principal  felt  had  benefited  most  from  the  program  in  the 
past — the  f  ifth  grades  (whose  teachers  wete  also  ';the  most'  enthusiastic 
about  CAI),  the  Special  Day  Classes,  and  the  children  in  the  Master  Plan 
program.     Children «would  go  to  the  lab  every  day  for  at  least  10 
minuteft.    Regular  fifth  graders  would  receive  the  math  curriculum, 
SDC  children  would  receive  math  and  rea.ding,  and  Master  Plan  children 
would  be  assigned  to  curriculums  according  to  their  needs.  After 
one  semester,  more  classes  would  be  added  to  the  schedule  if  possible. 

At  School  4  no  definite  plans  had  been  worked  out  at  the  time  of 
the  interview.    The  principal  expected  to  be  able  to  formulate  a  plan 
during  the  meetings  with  resource  teachers  after  the  close  of  school. 
The  principal  anticipated,  that  the  responsibilities  of  part-time  coordi- 
nation could  be  assumed  by  one  or  more  of  those  teachers.  Although 
the  principal  of  this  school  had  been  highly  supportive  of  the  CAI 
program  from  the  beginning,  and  although  the  program  had  been  regarded 
as  successful  by  the  participants  in  this  school,  the  school  and  community 
turmoil  of  1979-80  consumed  all  the  energies  of  the  principal  and  staff, 
and  it  was  not  possible  for  them  to  adequately  address  all  of  their  other 
important  concerns. 

The  two  small  non-Title  I  schools  shared  problems  quite  different 
from  those  of  Schools  3  and  4.    Schools  1  and  2  both  had  CAI  labs  with 
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only  enough  terminals  for  half  a  class.    The  absence  of  a  CAI  coordinator 
meant  that  a  teacher. must  take  the  entire  class  to  the  CAI  lab,  even 
though  only  half  of  the  children  could  work  at  the  terminals  at  one 
time.     This  had  been  tried  with  varying  degrees  of  success.     For  1980-81 
it  was  suggested  that  some  rearrangement  of  the  terminals  and  counters 
in  the  lab  might  make  this  a  more  workable  procedure.    Another  possibility 
was  io  have  teachers  work  together,  regrouping  children  so  that  those 
who  needed  ext^a  work' could  go  to  the  lab  with  one  teacher  while  the 
others  stayed  in  classrooms  covered  by  other  teachers. 

An  additional  problem  shared  by  Schools  1  and  2  was  the  lack  of 
access  to  funds  outside  the  regular  school  budget.    The  principal  of 
School  1  thought  that  funds  designated  for  a  library  aide  might  be 
used  for  a  CAI  aide  instead.     School  2 's  principal  planned  to  hire  a 
half-time  aide  using  some  carry-over  funds  from  1979-80.     School  1 
expected  to  have  their  former  CAI  coordinator  back  as  a  regular  classroom 
teacher  in  1980-81,  but  it  was  not  at  all  clear  how  they  would  be  able  to 
make  use  of  her  expertise  without  sacrificing  her  classroom  responsibil- 
ities, since  there  were  no  funds  to  coveri) additional  hours  for  her  as  a 
coordinator  or  "professional  expert." 

Neither  principal  could  specify  at  this  point  what  their  CAI 
program  would  look  like  in  1980-81.    Those  decisions  had  to  wait  until 
the  fall.    The  principal  of  School  1  expected  to  use  the  lab  for  older 
children  first,  adding  the  younger  ones  if  possible.    Preference  would 
be  given  to  teachers  who  were  enthusiastic  about  CAI,  because  the 
principal  felt  they  would  make  the  best  use  of  it.    The  principal  of 
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School. 2  planned  to  use  CAI  for  all  classes,  including  the  Special 

Day  Classes  and  the  adult  ESL  class  that  began  using  the  lab  in  1979-80. 

Thi:ough  all  of  the  discussions  of  future  use  of  the  CAI  program, 
the  problem  of  not  having  a  full-time,  certified  teacher  as  a  coordinator 
was  the  common  thread.    Without  a  doubt  the  coordinator  was  viewed  as 
the  critical  factor  in  a  successful  CAI  program,  but  the  principals 
were  determined  to  find  a  way  of  getting  some  good  out  of  the  program 
in  1980-81. 

r  . 

October,  1980 

Dr.  Ragosta  visited  the  four  CAI  schools  in  October,  to  learn  at 
first  hand  how 'the  CAl  programs  were  progressing.     The  situation  was 
worse  than  anyone  could  have  anticipated — there  were  no  programs  to 
observe.     The  LAUSD  budget  had  been  cut  by  $80,000,000,  and  there  was 

c 

absolutely  no  money  for  CAI  programs,  not  even  for  maintenance  or 
curriculum  rental.    Moreover neither  of  the  Bank  of  America  proposals 
had  been  funded. 

The  enormous  budget  cut  was  a  major  disruptive  fprce  in  all  the 
schools  not  only  because  of  its  effect  on  special  programs  but  also 
because  of  its  impact  on  class  and  faculty  size.    A  related  and 
equally  powerful  force  was  the  desegregation  plan,  which  drastically 
altered  both  the  total  enrollment  and  the  grade  makeup  of  many  schools. 
Because  the  desegregation  plan  was  being  redesigned  even  as  it  emerged, 
it  had  the  effect  of  keeping  schools  and  communities  in  a  perpetual 
state  of  uncertainty  about  what  new  arrangement  might  be  imposed  on 
them  tomorrow. 
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At  School  1,  the  desegregation  triad  that  had  seemed  to  work  well 
for  two  years  had  been  abandoned.     School  1  was  now  sending  its  first, 
third,  and  fifth  grade  students  to  another  school  and  receiving  a 
proportionate  number  of  second,  fourth,  and  sixth  grade  students  in 
exchange.     (In  past  years. School  1  kept  its  own  students,  in  grades  K-3, 
received  fourth  graders  from  two  other  schools,  and  sent  its  fifth 
and  sixth  graders  to  the  schools  from  which  the  fourth  graders  came, ) 
The  principal  had  no  money  to  do  anytW^ig- wirh^CAI^^ar^  that 
the  program  could  operate  at  all  in  1980-81 •  ^ 

At  School  2  the  situation  was  equally  gloomy.     The  principal 
expected  no  resources  of  any  kind  to  enable  the  school  to  run  a  CAI 
program. 

The  new  principal  at  School  A  was  interested  in  CAI  as  a  means 
of  lessening  the  "white  flight"  that  was  expected  to  result  from  the 
new  desegregation  arrangement  in  which  School  4  was  involved.    No  one 
was  quite  sure  just  what  that  arrangement  was  going  to  be,  and  a  lot 
of  conflicting  information  had  been  received.     Funding  for  CAI,  of 
course,  had  not  materialzed. 

The  principal  of  School  of  3  was  still  doggedly  pursing  funds  that 
might  enable  them  to  operate  the  CAI  lab  in  the  second  semester.  The 
principal  had  submitted  a  proposal  for  $5,000  to  add  to  $8,000  of  Title  I 
funds.    The  combination  would  provide  almost  enough  money  for  maintenance 
and  curriculum  rental. 

Los  Angeles  administrators  were  agreeable  to  Dr.  Ragosta's  idea 
of  submitting  a  proposal  to  the  Office  of  Special  Education  to  do  a 
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study  of  CAI  for  special  education  pupils*     The  possibility  of  removing 
the  equipment  from  Schools  I  and  2  and  installing  it  in  special  education 
schools  was  considered.     The  consensus  was  that  whatever  stood  the  best 
chance  of  reactivating  a  CAI  program  should  be  pursued* 

December,  1980 

As  expected,  no  CAI  program  existed  at  School  1  or  School  2, 
because  of  a  total  lack  of  funds*     The  two  schools  were  now  being 
served  by  one  principal  (who  had  previously  been  the  principal  of 
School  2),  the  former  principal  of  School  1  having  been  transferred 
to  another  part  of  the  district*     The  closing  of  School  1  was  again 
being  considered*     The  principal  had  been  advised  by  the  Area  4  office 
that  there  was  absolutely  no  hope  of  any  money  for  CAI  for  either 
school* 

The^^vrincipal  of  School  3  had  managed  to  secure  $12,000  in  discre** 
tionary  Title  I  funds,  which  he  supplemented  with  $1,611  of  School  3*s 
own  Title  I  money  to  make  up  enough  to  get  a  limited  CAI  program  into 
operation*  '  Because  School  3  qualifies  as  an  Educationally  Impacted 
School,  the  resource  specialist  who  agreed  to  serve  as  coordinator  of  the 
program  was  eligible  to  receive  an  11%  salary  increment  for  her  additional 
duties.    The  former  CAI  coordinator  at  School  3  returned  to  give  a 
two-day  Inservlce  program  for  teachers  and  the  new  coordinator.  The 
program ijwa8~--8e^up  to  serve  all  the  fifth  graders,  three  Special  Day 
Classes,  and  ten  children  In  the  Resource  Program,  a  total  of  154  children. 
Although  regretful  that  such  a  small  number  of  students  were  being 
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served,  the  principal  was  understandably  pleased  to  have  any  program  at 
all,  given  the  odds,  and  felt  that  it  was  running  well. 

School  4  received  a  similar  allocation  of;  Title  I  money  in  mid- 
December  and  began  making  arrangements  for  a  pirogram.    The  principal 
had  also  applied  for  a  special  education  grant  but  thought  there  was 
only  a  slight  chance  that  anything  would  be  forthcoming.     The  desegregation 
plan  had' been  put  into  effect  by  this  time,  so  that  School  4  contained 
first,  third,  and  fifth  grades,  and  the  school  with  which  School  4  was 
paired  consisted  of  second,  fourth,  and  sixth  grades.    The  principal 
mentioned  the  possibility  of  a  different  arrangement  for  next  year,  in 
which  School  4  would  become  a  magnet  school,  CAI  being  part  of  the 
"magnet.** 

March.  1981 

The  CAI  program  at  School  3  was  running  well,  serving  142  children 
in  regular  and  Special  Day  classes  and  about  25  in  the  Resource  Program. 
The  new  coordinator  was,  in  the  principal's  words,  "the  heart  of  the 
program.  ** 

The  CAI  program  in  School  4  began  operation  in  February.  There 
were  a  few  equipment  problems  and  some  difficulty  getting  repairs, 
but  by  March  everything  was  going  well.     Two  half-day  teacher  aides 
had  been  hired,  one  of  the  school's  regular  resource  specialists  was 
serving  as  a  coordinator,  and  the  original  CAI  coordinator  for  School  4 
came  every  other  Friday  to  help  in  whatever  way  she  could.     (She  was 
also  available  by  telephone  in  emergencies.)    All  classes  except  one 
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first  grade  were  participating  in  the  program,  including  the  Special 
Day  Classes.    The  teachers  and  students  were  enthusiastic. 

The  outlook  for  1981-82  in  both  schools  was  grim.     The  school 
district  budget  was  cut  by  an  additional  $15,000,000  over  the  previous 
year's  cuts,  and  funding  in  many  categories  was  likely  to  be  completely 
eliminated.     The  court  ruling  outlawing  bussing  to  achieve  integration 
was  to  take  effect  on  April  10,  1981,  but  with  parental  discretion; 
that  is,  parents  cWld  chose  to  have  their  children  return  to  the 
neighborhood  schools  or  l^ave  them  in  the  schools  to  which  they  were 
bussed.     The  last  nine  weeks  of  the  school  year  were  expected  to  be 
chaotic.    What  September  of  1981  might  bring  was  simply  impossible  to 
predict. 

June,  1981 

The  teachers  at  Schools  3  and  4  were  interviewed  at  the  end  of  the 
school  year.     School  3  had  used  all  three  curriculums  for  the  regular 
fifth  grade  classes,  rotating  curriculums  on  a  weekly  basis.     The  Special 
Day  Classes  tried  some  reading  and  language  but  used  mainly  mathematics. 
Each  class  went  to  the  CAI  lab  four  days  a  week,  for  15  or  20  minute 
sessions.     The  teachers  at  School  3  were  uniformly  enthusiastic  about  the 
program,     the  SDC  teacher  especially  so.     Several  teachers  mentioned  the 
advantage  of  being  in  the  lab  with  their  students,  so  they  could  see  where 
children  wCiTe  having  problems  "and  provide  follow-up  in  the  classroom.  They 
were  surprised  to  find  that  the  Spanish-speaking'  children  were  able  to  make 
progress  in  the  language  curriculum. 

J  1  " 


At  School  4,  only  the  mathematics  curriculum  was  us,ed,  and  every 
class  (except  the  one  that  did  not  participate  at  all)  went  to  the  lab 
twice  a  week.    Length  of  sessions  varied  from  15  to  30  minutes.    Of  the 
13  teachers  interviewed,  only  one  expressed  consistently  negative 
feelings  about  the  program.    The  other  12  were  glad  to  have  had  the 
program  in  their  school,  felt  thfeir  students  had  increased  their  math 
skills,  and  characterized  the  program  as  well-run,  successful,    and  non- 
disruptive  to  the  class.    There  were  various  opinions  about  scheduling, 
especially  with  regard  to  the  ideal  number  of  sessions  per  week  and  the 
choice  between  taking  half  a  class  to  the  lab  (leaving  the  teacher  %o 
work  with  a  smaller  group  in  the  classroom)  and  having  the  entire  class 
go  to  the  lab  at  one  time.    Most  of  the  first  grade  teachers  preferred 
the  latter  arrangement .    Three  teachers  said  they  would  have  liked  to 
have  used  fixed  strands  (concentrating  on  one  skill  at  ^  time)  instead  of 
variable  strands  (a  mixture  of  many  skills),  so  that  they  could  have 
coordinated  CAI  with  the  regular  classroom  work.    Two  teachers  who  had 
worked  with  CAI  in  previous  years  said  they  preferred  having  the  whole 
class  working  on  the  same  curriculum.    One  teacher  mentioned  that  she  had 
used  performance  in  the  CAI  lab  to  confirm  her  o\m  observation  of  children 
who  should  be  tested  as  possibly  gifted.    The  two  SDC  teachers  in  partic- 
ular reported  that  their  students  were  very  enthusiastic  about  the  program. 

The  funding  outlook  for  1981-82  was  even  vorse  than  the  T)reTious 
year.    Rising  costs  and  additional  budget  cuts  made  it  impossible  even  to 
imagine  where  money  for  a  CAI  program  might  be  found. 
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Log:    November  6,  1976  to  March  30,  1977  (Excerpts) 

Roberta  Woodson 
CAI.Room  Coordinator 
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LAUSD/C.A.I.  Project 


Roberta  Woodson 


November  6,  1976  LOG 


The  teminal  room  is  shaping  up.     It's  very  exciting.     The  counters 
are  in.    Painting  is  done — walls  yellow  and  doors  orange.     The  room  is 
ready  for  hardware.  Somehow  never  thought  it  would  really  happen. 

As  the  terminal  room  is  taking  shape,  students  are  showing  interest. 
Some  of  the  most  disturbed  kids  are  very  excited  about  the  computer 
coming.     It's  been  fun..to  show  them  the  process  of  a  classroom  being 
converted  to  handle  terminals  and  a  computer. 

When  the  workmen  are  sawing  or  hammering, . the  noise  disturbs  some  of 
the  classrooms. 

A  few  complaints. 


November  23,  1976 

The  terminal  room  now  has  its  mistress:    Clarissa  the  computer.  In, 
a  few  days  one  terminal  wilT  be  installed,  and  we  can  begin  working. 

Yesterday  was  the  first  CCC  Hardware  in-service,  at  School  3.  CCC 
personnel  directed  us  through  protected  routines.     It  is  very  fulfilling 
to  finally  work  on  hardware  tasks.    Anxious  to  begin  with  students. 


November  24,  1976 

Computer  repairman  here  today.     I  hope  computer  and  terminal  will 
soon  wotk.  Room  is  shaping  up~kids  come  by  and  ask  to  see  what's  going 
on,  they  seem  eager  to  start.     Next  Thursday  will  be  the  first  run-through 
of  the  schedule.    Teachers  are  interested  too. 

Will  be  ready  to  enroll  as  soon  as  ID//s  arrive  and  terminal  and 
computers  are  working 

My  Teaching  Assistants  are  excellent  and  full  of  enthusiasm  for  the 
project.     I  am  very  happy  with* both  of  them. 

\ 


-68- 


November  30,  1976 


Terminal  and  computer  are  UP-finally!    Have  enrolled  all  classes  but 
one.  Terminal  Is  doing  fine — pretty  logical.    Computer  printed  out  bad  I  ? 
for  ho  logical  reason. 


December'  2,  1976 

First  day  with  children!    All  day— very  interesting.    Fo]{irth-  and 
sixth-graders  had  many  questions:  /  . 

What  is  ID//?    Why?    How  does  computer  know  things? 

This  was  Just  a  general  introduction  to  the  room:    No  terminals. 
They  were  surprised, that  air  conditioning  was  needed  for  the  computer. 

One  second-grade  teacher  very  upset  that  the  groups  wer^ Vandomly 
selected,  because  the  grouping  was  in  conflict  with  her  math  grouping.  I 
felt  bad,  but  there  was  nothing  to  do  except  be  flexible.    In  terminal 
room,  some  groups  arrived  on  time,  others  not.    These  problems  will  get 
ironed  out. 

I  hope  the  10-15  minute  allotment  for  1/2  of  the  second-grade 
classes  will  be  enough  time  in  the  room. 


December  8,  1976  * 

Kids  all  day  agaife  worked  on  keyboards— which  keys  are  unique,  etc. 
Children  always  anxious  to  come  to  the  computer  terminal  room. 

One  class  of  4th-graders  seems  very  unruly— they  have  some  trouble 
walking  in  and  out.    They  stand  out  as  a  group. 

Students  keep  asking  when  the  rest  of  the  terminals  will  come. 

Ten-minute  time  makes  it  impossible  to  do  very  much  readiness  work 
with  those  second-graders. 


^  x 


-69- 


December  10,  1976 

One  teacher  is  still  upset  that  pull-out  is  random  for  her  second- 
graders*  She  does  not  complain  directly  to  me,  only  to  other  teachers.  I 
usually  hear  it  3rd-4th  hand. 

One  other  second-grade  teacher  was  disturbed  by  the  random  selection. 
Unders^tandably,  because  the  random  selection  at  Grade  2  really  wreaks 
havoc  with  the  teachers'  groupings. 

The  other  teachers  seem  very  cooperative;  they  seem  glad  to  be 
relieved  of  their  students  for  a  wjiile.     Strangely  enough,  not  too  many 
of  them  have  shown  ardent  interest  in  the  terminal. 

Two  teachers  so  far  and  the  assistant  principal  have  come  into  the 
room  and  actually  wanted  to  see  the  terminal  function. 

The  teachers  are  not  as  enthusiastic  as  the  students  are.    I  think 
this  will  change  when  their  students  become  more  involvedi^ 


January  3,  1977 

First  day  after  vacation.    All  32  terminals  lined  up  on  their 
counters.    Unfortunately,^  only  8  are  functional.     The  man  from  Cincinnati 

Millicron  is  here  working  on  the  computer  trying  to  fix  the  rest. 

Strange  to  see  32  terminal  CRT  screens  instead  of  32  smiling  faces. 

Lots  of  students  are  asking  when  will  they  start.    One  girl  asked 
"Do  we  Come  to  theo  incubator  room  today?" 

Many  teachers  asking  when  will  it  start?    Suddenly  realizing  CAl" 
will  permanently  be  in  their  schedule.     Another  second-grade  teacher 
upset  that  it  will  break  up  her  time. .  "What  will  I  do?" 

Fourth-  and  Sixth-grade  teachers  take  this  all  more  in  stride.  One 
says  that  CAI  doesbnot  take  the  place  of  Math.    She  ^suggests  I  remind  all 
the  other  teachers  of  that. 

Teachers  seem  a  little  uptight  about  scheduling. 

Fourteen  terminals  working  at  end  of  day. 


1'^  ^ 
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January  4,  1977 
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Still  waiting  for  computer  to  be  completely  up;  16  terminals  are  not 
working- 
Tomorrow  man  will  arrive  to  fix  Clarissa....  (I  hope!) 

Working  with  machines  is  sure  different  because  you  cannot  depend  on 
them.     Only  12  terminals  working  at  the  end  of  the  day. 

Several  people  came  by  to  see  the  installation. 

The  CAI  coordinator  from  School  2  came  over  very  frustrated — her 
computer  is  down  and  she  wants  action.    Even  more  impatient  than  I — didn't 
think  it  was  possible. 

CAI  start  up— probably  Fri.—- 1/7/77—- hope  so.     9§&ti^S  anxious.  So 
are  the  children. 


January  5,  1977 

Took  2  groups  of  2nd  graders  just  to  introduce  them. 

They  did  quite  well.     Some  trouble  with  finding  letters  of  their 
name  and  remembering  to  space  between  //  and  name.     Very  excited  students 
particularly  when  their  last  name  appears  on  the  screen.     They  said— **He^ 
knows  me!"    "How  does  he  know  my  name?" 

Only  14  terminals  working  this  a.m. 

11:00  a.m.; — Cincinnati  Millicron's  "John"  here  to  work  on  the 
computer.    Also  Ray  from  Hazeltine  working  on  line  printer. 

The  coordinator  from  School  1  is  very  low.    Morale  is  hard  to  ^eep 
up  when  your  system  is  down.  / 


Having  to  postpone  start-up  time  hasn't  affected  teachers,  as  yet. 
Hopeful  we  can  start  up  in  the  next  few  days — I'm  not  sure,  though. 


/ 


/ 


/ 
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January  7,  1977 

Yesterday  and  today—introduced  all  2nd-graders  to  the  terminals. 
All  in  all  went  very  well.     Students  very  excited  to  see  their  last  name 
on  the  screen. *  Spanish-speaking  students  were  equally  excited  as  the 
others.    My  very  bilingual  TA  explained  procedures  to  them  also.  They 
put  in  their  numbers  and  names  very  well. 

Starts  up  is  definitely  on  Monday  for  all  grades.     The  Ath-6th  grade 
students  have  had  no  introduction—we'll  see  how  everyone  does. 

Students  Just  walk  in  and  stare-^ery  excited. 

Two  more  teachers  were  in  to.day  and  are  interested  in  seeing  how  the 
terminals  work. 


January  10,  1977 
We  started! ! 

Very  rushed—problems  with  numbers  and  names— 
-   Problems  with  <  and  >.  ' 

Trouble  due  to  newness  of  working.  Time  will  probably  take  care  of 
lots  of  the  problems. 

The  lessons  are  a  full  10  and  20  minutes.  That  plus  the  newness  of 
signing  in  caused  our  schedule  to  run  late  all  day.  As  the  students  get 
used  to  it,  they  will  function  more  quickly* 

Students  were  excited. 

A  few,  after  working  a  very  full  20  minutes,  were  very  tired. 

<;^.      I  have  still  not  mastered  learning  about  the  terminals.     If  kids 
press  certain  buttons,  terminals  go  off.     I  will  continue  to  look  into 
all  those  things. 

We  should  have  introduced  students  to  the  shift  key  before  we 
started— especially  for  the  greater  than/lesser  than  s)rmbols. 


12  i 


January  11,  1977  ^.  ^  ' 

'  •  /        _  ■  ■  "  . 

;  Much  better  today.  / 

/    Almost  correctly  on /schedule,  and  many  people  knew  how  to  sign  in 
accurately.  / 

/  /  ,  . 

I      Still  some  problem^. 

Spanish  speakers  nave  lots  of  trouble  signing  in.    Also  cannot  read 
the  Reading  or  Languag^e  curriculums.    They  <io  much  better  in  math  curriculum. 

Staff  Meeting;    /^blic.    One  second-grade  teacher  very  upset.  Feels 
that  reorganization  has  so  drastically  affected  her  instructional  program, 
she  doesn't  know  wha^^  to  do.     She  doesn't  like  students  pulled  out  of  her 
class  randomly.     Sh^  &aid  it  takes  her  hours  to  reorganize  her  program 
around  the  kids  who/  leave  for  their  computer  lesson. 


Today,  with  all  terminals  functional,  I  feel  more  aware  of  how  to 
solve  minor  problems.    Students  feel  more  successful  today.  They're 
aware  of  their  percentile  scores.    Four  students  received  100%  today.  I 
am  beginning  a  club.     Students  may  join  if  they  get  ten  100%  scores. 

Very  pleased  with  second  day.    Hope  the  project  continues  to  improve. 


January  12,  1977 

/ 

More  noise. 

/ 

Students  very  high.     Second-graders  still  have  some  trouble  with 
name  and  number.    Particularly,  Spanish  speakers.    Also  classes  must  be 
on  time  up^ to  the  terminal  room  or  it  really  gets  the  schedule  off. 

/     -  *  .    ■        ■  ■ 

4th-6th  graders  did  much  better  today.    Most  of  them  much  more 

fluent  'ixL  signing  oh  and  off. 

Today,  only  two  lOOXers.    One  girl  has  topped  out  of  the  laijguage 
program/  already.    Put  her  into  math  curriculum. 

Students  must  be  punctual  in  order  for  all  to  go  smooth. 

/  ■  ^.  . 

Also  have  to  get  our  routines  more  functional, , too. 

/'  ,  '    -  " 

/ 

/Ray  from  Hazeltine  is  excellent.  Calls  in  to  see  how  everything  is 
doing.    Very  good  maintenance. 

i 


f 
7 


January  13,  1977 


/ 


Today  was  very  smooth — in  every  way— all  students  were  punctual — 
more  fluent  in  signing  in.    About  six  got  100s;  kids  more  excited. 


January  17,  1977 

Went  pretty  well.    Kids  high. 

Beginning  to  make  a  list  of  Spanish-speaking  kids  that  need  to 
switch  their  reading  or  language  treatments  to  math. 

More  100s  than  ever  today~about  16  or  17— mostly  6th-graders. 
Some  6th-graders  very  frustrated  and  yelling  at  machine — some  will  not 
try  to  be  accurate.    Worked  more  on  coming  in  and  leaving  the  terminal 
room  more  quietly. 

Tomorrow  is  TV  filming—should  be  interesting.  " 

Visitors  today.     Stayed  20  minutes.    One  said  that  I  run  this  room 
with  an  iron  hand.    Two  4th-  grade  teachers  said  it  really  is  nice  when 
1/2  class  leaves  for  GAI.     They  say  the  computer  program  is  going  very 
well.  ^ 


January  19,  1977 

Today— computer  stuck  again — so  lost  some  lessons  (the  10:00  a.m. 
group).    Yesterday  same  thing  happened  at  2:20  or  so.    We  fixed  it,  but  I 
did  call  CCC.  Kids  excited  today.     My  back-up  was  working  in  CAI  room 
today.    Now  she's  trained,  in  case  I'm  not  here. 


January  20,  1977 

Teachers  of  4th-6th  grade  complain  that  their  kids  are  leaving  and 
entering  rooms  quite  noisily.     Need  some  escorting. 

Most  kids  just  buzzing  along.     Some  seem  distracted  and  have  very 
short  attention  spans.    There  have  been  fewer  100s. 


-A,  ^/ 


January  20,  1977  (continued) 

o  - 

Changed  Spanish-speakers  to  math  treatments* 
Also  top-outs  were  enrolled  in  math* 


January  21,  1977 

Second-graders  beginning  to  independently  put  in  second  course  very 
well*    At  9:45—10:00  a*m*     "Computer  Errors**  appeared  on  some  terminals* 
I  called  CCC*    Had  to  cancel  classes  for  the  day*    Luckily  we  run  back-ups 
daily*    So  we  may,  only  lose  today's  work— I  hope* 

Computer  up  in  afternoon* 

All  is  OK* 


January  24,  1977 

Spanish  4th-graders  who  had  treatments  switched  to  math  are  dping 
much  better*    Frustration  level  down*    Scores  over  90%* 

Also  started  noting  down  facts  for  teachers*    Makes  schedule  more 
hectic*    But  I'm  enjoying  getting  more  specific  about  student's  needs* 
,Also,  I'm  so  glad  the  Spanisli-speakers  are  less  frustrated— getting  99% 
and  seem  happier*     TV  outfit  is  still  promising  to  arrive  but  has  not 
appeared  yet* 

Later  today  is  the  teacher  in-service  for  C*A*I*    I'm  both  scared 
and  looking  forward  to  it** 

The  TV  people  came*    We'll  be  on  the  news  possibly  between  5-6 
tonite*    Computer  went  down  in  the  middle  of  videotaping! 

In-service  pretty  good—but  computer  errors  just  before  we  began* 
Very  frustrating* 

Mlliicr on  man' coming  tomorrow  a*m*  to  check  soft  disc  error*  So 
awful— I  had  to  cancel  classes — I  guess  the  machine's  limitations  are 
really  get;ting  to  me*    We  lost  the  last  two  groups  today* 
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January  26,  1977  ' 
Computer  Down! 

January  27,  1977  - 

During  in-service,  some  teachers  felt  that  horizontal  addition  and 
subtraction  were  not  good  for  students — does  it  deemphasize  place  value? 
Wanted  that. comment  brought  to  ETS's  attention. 

Also  felt  language  program  didn't  leave  enough  time  with  answers. 
They  flash  the  answer  on  the  screen  too  quickly — 

Computer  still  down! 

Having  big  conference  at  School  2.    CCC,  Hazel tine  and  Cincinnati 
Millicron — Please  fix  what's  going  on. 


January  31,  1977 

Conferenced  w/all  teachers 


February  1,  1977 

Down  in  a*m.    Some  classes  in  p.m. 


February  2,  1977 

•  Down  again  this  a.m.  Very  frustrating.  Serviceman  comes  and  fixes 
the  machine  and  within  a  few  minutes  it  does  not  work  again.  I  wish  the 
machine  could  be  fixed  once  and  for  all.  Computer  errors  keep  appearing 
and  reappearing.    When- will  It  end? 

School  2  is  still  down. 


-Co 
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February  3,  February  4,  1977 

Up  and  running.  Very  smoothly.  Started  to  record  X's  every  day  for 
all  6th-grader8  and  some  2nd-graders«  Then  charting  their  weekly  average 
scores. 

Kids  are  very  motivated  about  their  scores  improving.     I  think  it's 
good  to  keep  score  even  though  its  only  the  100%ers. 


February  1\  1977 

Down  again — extremely  frustrating. 


February  19,  1977 

Two  6th-graders  have  bad  attitudes  toward  C.A.I.    One  has  math — she 
keeps  asking  for  another  course.     (R-L)    She  gets  very  discouraged — left 

in  the  middle  of  a  lesson.    Too  bad!    Also  had  trouble  handling  herself 
in  a  regular  classroom. 

Most  6th-graders  are  very  attentive — also  4th-graders.  Interested 
to  see  how  they  do  tomorrow.  t 


February  18,  1977 

Note;     Students  in  4-6  better  behaved  during  first  session.  By 
second  session — a  little  more  mumbling. 

Today  with  the  new  30-second  timeout  replacing  the  60-second  timeout, 
there  was  lots  of  math.     Students  very  frustrated,  especially  2nd-graders; 
it  will  take  them  awhile  to  get  used  to  having  less  time  to  do  their 
work.  * 

(. 

Second-graders  now  have  7-minute  sessions  instead  of  10.    First  day 
today.    All  of  them  were  particularly  affectionate  today  (lots  of  hugs  and 
kisses).    I  wonder  if  this,  was  related  at  all  to  shorter  sessions — we'll 
see. 

One  2nd-grader  turns  off/on  line-button  to  off  to  avoid  getting 
timed  out.    8mart~huh?    Another  4th-grader  figured  out  Control  Z.  Very 
strange.     [Note:    Control  Z  is  a  method  used  to  stop  a  session  before  the 
time  is  up. J    Both  are  very  hyperactive  and  have  behavior  problems. 
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February  25,.  1977 

Rainy  days — two  in  a  row— seem  to  affect  behavior  in  CAI  room  as 
well  as  the  rest  of  the  school.     Last  2  days  kids  have  been  more  anxious 
and  talkative.     Scores  even  went  down^ a  bit.     Concentration  level  lower. 

It's  so  nice  now  that  CPU  and  terminals  are  working  all  the  time — for 
2  solid  weeks.  .  - 

This  enables  us  to  move  into  areas  of  helping  students  and  teachers. 


March  2,  1977 

As  all  machines  are  working  better,  this  frees  me  to  be  more  motivat- 
ing to  the  students — developed  100%  club  for  R  &  L;  85%  for  M,  since  it's 
a  little  more  difficult.     Students  seem  more  motivated  by  this  move,  and 
more  want  to  come  at  8:30  a.m.*  and  put  in  make-ups  or  demos.  Also, 
6th-graders  want  to  do  demos  after  school  in  lessons  they  don't  have 
regularly. 


March  4,  1977 

Two  kids  asked~how  about  being  able  to  rub  out  in  math?  [Literally 
erasing  the  numbers.] 


March  9,  1977 

Parent  conferences  today — kids  bringing  their  parents  to  the  CAI 
room — very  exciting  because  children  are  enthusiastic  about  what  they  are 
doing. 

85%  math  is  pushing  more  students  into  getting  better  scores—also 
now  taking  attendance  daily — I  will  do  awards  for  perfect  attendance.  So 
everyone  can  feel  comfortable  receiving  something.     Now  I  want  to  begin 
to  turn  my  thoughts  toward  low  achievers — 

Kids  consistently  scoring  under.  70,  what  to  do?? 


March  10,  1977 


Program  is  in  very  good  shape  right  now. 

e-ups  every  a.m.    Kids  very  enthusiastic. 

I  find  time  to  teach — al?  3  of  us  spend  time  with  contact  (physical, 
like  patting  on  the  back) »  touching.     It  calms  the  students— -they  feel 
more  like  working.     It's  important  to  them  to  know  people  are  available  to 
them  as  well  as  the  machines.     I  work  with  some  kids  after  school,  going 
over  concepts  they  find  hard  to  understand. 

Feel  time  has  conie  whetn  I  can  be  more  personally  involved  with  kids 
and  where  they're  at — jacademically  and  "af f ectively"~I  always  thought 
the  Job  would  become  bbring  after  awhile.  I  saw  a  lot  of  coordinators 
sitting  around  in  some i of  the  other  CAI  installations  I  visited,  but  I 
doubt  that  I  ever  wllli  Students  have  too  many  questions-- there 's  too 
much  to  teach  them.       1  > 

One  teacher  is  presenting  the  only  problem—she  feels  CAI  interrupts 
her  instructional  time~8o  she  has  many  times  forgotten  to  send  her  group 
until  late-— she  also  maintains  that  all  special  programs  scheduled  her 
students  at  the  worst  possible  time. 


March  30,  1977 

After  Easter.    Now ^twlce  a  week  for  one  hour  I'm  alone.    Room  takes 
on  a  dif ferent '  feel~I  must  be  more  the  "custodian"  of  the  30  students 
during  this  time  rather  than  the  teacher.    But  not  too  bad — : 

We  have  had  2  days  when  subs  relieved  the  teachers  for  short  periods 
to  visit  the  CAI  room.     It  was  good.    Teachers  got  more  motivated  about 
the  program.     The  CCC  in-service  helped  some  teachers  too. 

I  wonder  how  the  end-of-year  testing  will  affect  us. 


J  3^ 
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';  Appendix  B 

1 

i 
I 

\  Thoughts  on  a  CAI  Program: 

j 

CAI  Coordinators  Debriefing  Conference 
June  8-12,  1980 
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THOUGHTS  ON  A  CM  PROGRAM; 
CAI  COORDINATORS  DEBRIEFING  CONFERENCE 


June  8-12,  1980 
Montecito,  California 


Marjorle  Ragosta* 
in  consultation  with  CAI  coordinators: 
Mary  King 
Marjorle  Lord 
Judy  Newman 
Ann  Vasilopoulos 
Rayma  Wells 
Roberta  Woodson 


*  The  help  of  Teri  Mondeaux,  Puff  Rice  and  Glna  Wilson  must  also  be  acknowledged. 


Educational  Testing  Service 
Princeton,  New  Jersey 
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"It* 8  a  crime  to  have 
the  QAl  lab  in  use 
less  than  full  .time.' 


The  Coordinators 


PREFACE 


The  ETS-LAUSD  CAI  Project  ceased  operations  in  elementary 
schools  in  Los  Angeles  on  June  30,  1980.    For  4  years  funding  from 
the  National  Institute  of  Education,  direction  from  Educational 
Testing  Service,  and*"  the  tremendous  work  of  personnel  in  the  Los 
Angales  Unified  School  District  combined  to  operate  a , uniquely 
successful  project.'   During  the  1980-81  school  year  the  LAUSD  and, 
specifically,  the  4  schools  with  CAI  labs  will  try  to  carry  on  the 
program  without  the  government  funding  which  produced  and  supported 
the  CAI  labs.    The  loss  of  CAI  funding  will  meat  no  full-time  CAI 
coordinators  in  the  schools;  in  fact,  most  of  the  CAI  coordinators 
will  have  moved  away  from  the  CAI  schools  by  the  1980-81  school 
year.    With  money  and  CAI  personnel  gone,  schools  will  have  to 
rely  on  their  own  staffs  to  organize  and  manage  the  CAI  program. 

It  is  not  an  iiiq>ossible  task.    In  interviews  in  June  1980, 
most  teachers  in  Ci^I  schools  had  positive  feelings  about  bringing 
their  classes  to  the  CAI  labs  on  their  own — once  they  had  received 
8or}e  in-service  training  on  the  computers  and  terminals.  The 
perceived  success  of  the  CAI  program  during  its  four  years  of 
operation  laid  the  foundation  for  the  successful  transition  of  CAI 
from  a  federally  funded  project  to  a  school  program.    In  ah  effort 
to  help  that  transition  and  to  perpetuate  the  knowledge  and  expertise 
gained  by  CAI  coordinators,  this  report  has  been  written.    To  the 
extent  that  it  meets  the  needs  of  those  of  you  who  will  function  in 
the  CAI  labs  in  the  coming  years,  thanks  are  due  to  those  who  in 
the  last  4  years  made  CAI  work:    our  CAI  coordinators.  They 
were  a  dedicated  lot.    They  pass  their  information  on  to  you  with 
the  hope  that  computer-assisted  instruction  will  continue  to  help 
students. 
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Resource  Peqple 


The  following  people  have  given  their  permission  to  be  contacted 
for  help  if  needed. 


Harjorie  Lord^ 

(4  years  at  School  1) 

0 

Judy  Newman 

(4  years  as  substitute) 


Phone  Number 


(213)  392-1086 


(2l3)  456-3727 


Ann  Vasilopoulos 

(4  yeti.2  8  at'  School  2) 


Rayma  ;:lls 

(3  ye  rs  at  School  3) 


(213)  474-9869 
(213)  450-2343 


!;  ;  oerta  Woodson 

(3  years  at  School  4) 


(213)  559-8121 


Marjorie  Ragosta 
Educational  Testing  Service 
Princeton,  N.J.  08541 


(609)  734-5702 


NOTE:    Computer  Curriculum  Corporation  provides  people  for  in-service 
training  as  part  of  its  contract. 

^        "  The  CCC  Proctor's  Manual  contains  information  on  hardware 
procedures  and  routine  problems. 

.  --     .         ■     '  •       x-f  /  ■ 

The  Teacher  Manuals  from  CCC  provide  information  on  'the  ^ 
three  CAI  curriculums:    Mathematics^  Strands,  Reading^  rjtnd 
Language  Arts. 
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CURRICULUM  SECTION 
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The  CAI  Curriculums 

Full  descriptions  of  the  CAI  curriculums  in  Mathematics 
Strands »  Reading,  and  Language  Arts  are  available  as  teachers* 
handbooks  from  Computer  Curriculxim  Corporation  in  Palo  Alto, 
California.     Each  of  the  curriculxims  is  composed  of  several  strands, 
i«e*»  sequences  of  related  items  such  as  vertical  addition,  horizontal 
addition,  measuremen<|:,  etc. ,  in  the  Mathematics  curriculum.  The 
strands  structure  allows  each  student  to  move  at  his  own  pace 
independently  in  each  strand  of  the  curriculxim.    In  the  mathematics 
curriculum  problems  can  be  presented  to  students  in  a  mixed 
drill— called  variable  strands— or  in  a  fixed  mode  within  one 
strand.    In  the  language  arts  curriculxim  questions  may  be  presented 
to  students  in  two  ways:    variable  strands  (mixed  drill)  or  topics 
(single  topic  drill).     The  variable  strand  approach — in  mathematics, 
language  arts ,  and  reading — is  automatic  and  requires  only  one 
initial  enrollment.    Fixed  strands  in  the  mathematics  curriculxim  • 
and  topics  in  tiie  language  arts  curriculum  require  simple  procedures 
whenever  a  specific  assignment  is  made  or  changed. 

Coordinators  were  asked  to  rank  the  CAI  curriculums  and  their 
alternative  methods  of  use.    'They  ranked  six  alternatives:  the 
Mathematics  curriculxim  as  (1)  variable  strands,  (2)  fixed  strands 
or  (3)  a  combination;  the  Language  Arts  curriculxim  as  (4)  variable 
strands  or  (5)  topics;  and  (6)  the  Reading  curriculum.    The  curricu- 
Ixims-were  ranked  from  1  (the  best)  to  6  (the  least  helpful). 
Results  are  given  in  Table  1.  ' 


Table  1 


Curriculum  Rankings 


■  • 

COORDINATOR 

CURRICULUM 

AVERAGE 

.a) 

(2) 

(3)  (4) 

(5) 

(6) 

RANK 

1.5 

1 

1 

1 

3 

2 

mathematics  (variable  strands) 

1.J6 

4 

3 

2 

3 

2 

3 

mathematics  (fixed  strands) 

2.8 

1.5 

2 

3 

2 

1 

1 

mathematics  (fixed  +  variable) 

1.8 

5 

5 

5 

4 

5 

5 

language  (variable  strands) 

4.8 

3 

4 

4 

5 

4 

4 

language  (topics) 

A.O 

6 

.  6 

6 

6 

6 

6 

reading  ^ 

6.0 

Among  coordinators  there  was  general  agreement  that  the 
mathematics  program  was  best,  language  was  next,  arid  the  reading 
curriculum  ranked  last.    Within  the  mathematics  CAI  curriculum, 
variable  strands  was  preferred  although  the  combination  of  variable 
and  fixed  strands  received  almost  an  equal  rating.    Within  the 
language  curriculum,  topics  was  preferred  over  strands.  Specific 
comments  on  each  of  the  six  methods  of  using  CAI  are  presented  on 
the  next  two  pages.  '  < 

"  c  ■  •  . 
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The  Mathematics  CAI  Curriculum 

Mathematics  variable  strands*    This  Is  suitable  for  most 

students  most  of  the  time  to  keep  Interest  and^ 
attention  high.     It  requires  no  extra  work  on 
the  part  of  teachers.    Research  has  shown  it  to 
Ije  effective. 

Mathematics  fixed  strands.    This  is  especially  good  for  reln- 
for.cement  after  a  specific  skill  is  taught  in 
class.     It  is  helpful  to  students  with  specific 
weak  areas,  for  SDC  students,  for  students  in 
primary  level  of  success.     Its  specific  weakness 
is  that  it  requires'  less  concentration  and  tends 
to  be  boring  for  continuous  use. 

Combination;    mathematics  fixed  and  variable.    This  was  used 
very  successfully  at  the  second-grade  level  and 

> 

was  felt  to  be  appropriate  for  all  classrooms  at 
all  grade  levels.     A  good  description  of  how  the 
combination  of  fixed  and  variable  strands  can  be 
used  is  given  in  the  CAI  program  for  grade  1  on 
page  6. 

The  flexibility  of  the  mathematics  curriculum  allows  teachers  to 
decide  how  it  might  best  be  used  to  satisfy  the  individual  needs  of 
the  students  in  the  classroom.     There  is  no  necessity  to  choose  one 
program  for  the  whole  class;  the  computer  can  easily  be  made  to, 
adapt  to  the  special  needs  of  the  individual  members*    However,  it 
does  require  some  work  on  the  part  of  teachers  to  come  up  with 
programs  best  tailored  to  individual  students. 
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The  Language  Arts  CAI  Curriculiim 

Language  strands.    This  was  the  language  curriculum  userd  for 
the  CAI  study.     It  is  useful  for  most  students 
in  grades  3-6,  although  its  range  is  much  more 
.restricted  than  in  the  mathematics  curriculum. 
Bright  fifth-  or  sixth-grade  students  may  top 
out  of  the  program,    Spanish-speaking  students 
(NES-LES)  may  not  be  able  to  understand  the 
program.    Some  transitional,  limited-English- 
speaking  students  feel  little  threat  in  using 
the  computer  and  can  acquire  the  technique  and 
vocabulary  to  work  in  this  curriculum  with 
surprising  success. 

Language  topics.    Almost  all  caordinators  ranked  topics  higher 

than  variable  strands.    It  is  useful  in  reinforcing 
skills  taught  in  the  classroom  or  for  students 
who  need  drill  in  weak  areas.    It  works  very 
well  when  the  teacher  can  coordinate  it  with 
classroom  activities.     It  requires  more  time  on 
the  part:  of  teachers  to  determine  the  specific 
topics  to  be  used  and  to  access  them  on  the 
computer. 

The  Reading  CAI  Curriculum 
^  Reading,  Movement  within  the  curriculum  ia  slow  and 

frustrates  some  students.    Others  enjoy  it. 
It  was  ranked  lowest  by  all  of  the  coordinators, 
although  the  research  seems  to  show  that  it 
helps  to  build  vocabulary, 
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A  CAI  Program  for  Grade  1 

For  pupils  in  the  elementary  grades'—especially  grades  1  and  2 — 
only  the  mathematics  CAI  curriculum  is  available.  Therefore, 
suggestions  for  grade  1  need  very  little  modification  to  be  suitable 
for  grades  2  and  3. 

Readiness  Activities 

First  graders  need  some  readiness  experience  before  starting 
their  CAI  program.    Children  need  an  introduction  to  the  terminal 
keyboard  and  practice  in  signing  on.    Experience  with  demonstration 
lessons  (DEMOs)  is  helpful.    Until  children  become  familiar 
with  the  proceiBS,  it  is  useful  to  have  a  3  x  5  card  with  the 
information  needed  for  each  child  to  sign  on.     This  includes  the 
child's  ID  number,  first  name,  and  the' letter  necessary  to  access 

either  variable  math  strands  (M)  or  fixed  strands  (F).  Teachers 

1 

and  aides  can  assist  small  groups.    Cross-age  tutors ,  experienced 

1 

with  jCAI,  could  be  very  helpful. 

I 

1  ,  .  " 

|First  graders  will  also  need  help  wi^th  the  (very  limited) 

i  , 

reading  vocabulary  required  for  the  mathematics  curriculum,  including: 

I     Number  and  name,  please       Ones  Pennies 
Course  Tens  Nickles 

I    Type  Count  by  Dimes 

How  many  i^'s 

Vocabulary  in  the  first  column  is  encountered  in  the  early  lessons. 

Children  also  need  to  know  the  CAI  lab  standards  of  behavior 
in  ord^r  to  get  the  maximum  benefit  from  their  CAI  experience. 


er|c  ,  .      :  _  11: 


The  CAI  Schedule 

.  .    Depending  on  the  Individual  school  situation,  first-grade  . 
classes  might  be  assigned  to  the  CAI  lab  for  either  semester  or  both. 
One  lO-ininute  session  dally,  coming  before  or  after  a  natural 
break  In  the  school  day,  was  suggested  by  all  coordinators • . 

In  the  absence  of  a  full-time  CAI  coordinator,,  teachers  would 
bring  their  classes  to  the  CAI  lab,  provide  help  to  the  students  In 
the  lab,  and  leave  the  lab  with  .their  students.    Help  would  be 
provided  whenever  possible  by  a  teaching  assistant  assigned  to  the 
CAI  lab.    Additional  help  could  be  provided  by  the  classes*  own 
teaching  assistants  in  Title  1  schools. 

In  the  schools  with  small  CAI  labs  only  half  of  the  class  at  a 
time  will  be  able  to  work  at  the  terminals.     Suggestions  were  made 
to  have  two  or  three  banks  of  terminals  and  one  or  two  banks  of 
counter  space.    Independent  work  requiring  a  minimum  amount  of 
direction,  materials,  or  supervision  was  recommended  for  the  group 
not  at  terminals.    One  coordinator  (in  a  large  lab)  said: 

I  would  pull  small  groups  of  children  off  the 
terminals  to  teach  to  specific  needs  as 
indicated  by  the  CAI  printout,  then  follow  up 
with  fixed  strands  in  the  problem  area. 

A  CAI  Program  for  Grade  1  continues  on  the  next  two  pages. 
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Curr.lcul\m  Options 

Coordinators  were  equally  divided  in  their  choice  of  variable 

or  fixed  strands  for  the  students'  initial  entry  into  CAI;     In  actual 
practice  in  the  CAI  study,  the  variable  strands  approach  was  used  in 
grade  1.    Although  all  of  the  Coordinators  had  been  initially  appre- 
hensive at  the  thought  of  first-grade  pupils  starting*  variable  strand 
CAI  within  five  weeks  of  entering  school,  the  first  graders  adapted 
readily  to  the  program.    The  single  complaint  was  that  some  children 
were  going  too  fast!    One  K-1  teacher  reported,  "In  my  17  years  of 
working  with  K-1  students,  this  is  the  first  year  we  have  finished  the 
book.    I'm  sure  it's  because  of  the  machines."    The  variable  strands 
approach  has  been  shown  to  be  an  effective  program. 

All  Coordinators  agreed  that  weekly  CAI  printouts  be  used  as  a  guide 
in  assigning  students  to  fixed  strands  for  individual  progress,  for  extra 
help  in  weak  areas,  or  for  reinforcement  of  skills  taught  in  the  classroom. 
Two  coordinators  suggested  fixed  strands  for  the  first  semester  followed 
by  variable  strands  for  the  second.    Others  suggested  fixed  strands  once 
or  twice  a'week,  with  variable  strands  carrying  the  major  responsibility 

for  the  CAI  program. 

One  Coordinator  wrote: 

I  would  prefer  to  have  my  first-grade  class  use  the  CAI  lab* 
just  before  lunch  for  daily  lO-minute  sessions  of  math.  I 
would  want  them  assigned  to  Math  variable  strands  first. 
After  the  rapid  motion  of  the  first  ten  lessons  I  would 
study  the  report  to  determine  which  children  needed  to  be 
placed  in  fixed  strands  and  which  fixed  strands  to  assign. 
Using  the  child's  average  score,  it  is  easy  to  see  where,  or 
if,  he  is  in  need  of  extra  drill  in  a  particular  strand  or 
strands.    I  would  be  likely  to  have  the  children  work 
Monday,  Wednesday,  &  Friday  on  variable  strands  and  Tuesday  & 
Thursday  on  fixed  strands.    A  report  every  Friday  will  show 
whether  the  fixed  strand  needs  to  be  changed.    If  the  child 
shows,  no  progress  he  needs  special  help  from  the  teacher— if 
he  has  made  rapid  progress  he  is  ready  for  a  different 
strand.    Record-keeping  for  each  child  is  important  in  order 
to  know  which  strands  have  been  assigned  and  for  how  long. 


Motivational  Hints 


Initially  students  are  highly  motivated  to  work  and  to  get 

good  scores  in  CAI,    Later  the  work  gets  more  routine,  and  a 

little  planning  is  helpful  in  keeping  up  motivation.    The  following 

methods  have  been  found  useful: 

•  Awards*    Commercial  awards  available  in  ditto  masters 
were  used  for  scores  of  80%  or  90%  or  better. 
Teacher-made  awards  were  also  used,  some  of 
which  are  included  at  the  end  of  this  report. 
At  least  once,  a  student-designed  award  was 
used. 


Contests.    Contests  were  run  with  classrooms  competing  . 
against  one  another.    Points  were  earned 
for  each  student  in  the  class  who  scored  at  a 
certain  level  or  above.     Some  contest  materials 
are  also  included  at  the  back  of  this  report. 

Line  leaders.    The  boy  and  girl  who  got  the  highest ' 

number  of  correct  answers  were  line  leaders  on 
.the  way  back  to  the  class. 

Applause.     For  the  highest  number  correct— or  the  highest 
number  attempted— students  were  occasionally 
applauded  by  the  other  students. 

Manipulatives.     For>students  with  difficulties  in 

adding  or  subtracting,  manipulatives  are  helpful. 
One-inch  blocks  are  recommended  because  they 
don't  get  into  the  keys  of  the  terminals. 

Record  keeping.    Keeping  a  score  sheet  to  monitor  one's 
own  progress  was  motivating  to  many  students. 
^        Stickers  or  stamps  for  high  scorers  helped. 

CAI  reports.     Some  students  really  enjoyed  pulling 
their  own  CAI  reports  from  the  computer. 


-93- 

A  CAI  Program  for  Grade  5 

For  students  in  the  upper  elementary  grades  all  three  of  the  CAI 
currlculums  are  available:    mathematics,  language  arts/  and  reading. 
With  little  or  no  modification  a  program  suitable  for  fifth-grade 
students  would  also  be  suitable  for  students  in  graues  4  and  6. 
Readiness  Activities 

Experience  with  demonstration  programs  (DEMOs)  is  helpful  to 
students.    It  is  also  useful  to  prepare  3x5  cards  with  the 
information  needed  by  each  student  to  sign  on:    the  ID  number, 
the  student's  first  name,  and  the  letter(s)  necessary  to  .access 
the  CAI  curriculums  which  have  been  selected. 
The  CAI  Schedule 

There  was  remarkable  unanimity  among  Coordinators  as  to  how 
they  would  use  CAI  if  they  were  fifth-grade  teachers  with  classes  of 
30  children.    Everyone  chose  to  take  the  entire  class  to  the  CAI  lab 
for  a  twenty-minute  session  every  day.    One  Coordinator  suggested  that 
three  times  a  week  would  be  worthwhile  if  that  was  all  that  was  possible 

Two  Coordinators  mentioned  the  desirability  of  arranging  the 
schedule  so  that  the  class  went  to  the  CAI  room  in  connection  with  a 
natural  break  during  the  school  day.    One  felt  that  it  was  important 
to  avoid  interrupting  the  long  reading  period  in  the  morning. 

One  Coordinator  said  she  would  certainly  schedule  the  CAI 
work  at  a  time  when  her  teacher  aide  could  go  with  her,  so  that 
there  would  be  three  adults  in  the  lab  with  the  children  (assuming 
that  a  TA  were  assigned  to  be  in  the  CAI  room  regularly).  Another 
person  emphasized  the  importance  of  recruiting  parent-  volunteers 
and /or  student  tutors. 
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Curriculum  Options  ^ 

oAll  of  the  coordinators  chose  to  ube  the  mathematics  and 
 language  CAI.  curriculuma#    -   • 

Most  of  the  coordinators  said  they  would  use  a  combination  of 
variable  and  fixed  strands,  with  variable  fetrands  serving  as  the 
core  of  either  the  mathematics  or  language  curriculum.  Students 
would  be  assigned  to  fixed  strands  in  math  or  language  topics  for 
some  lesser  amount  of  time,  either  for  remedial  work  as  needed  or 
for  reinforcing  specific  lessons  taught  in  the  regular  classroom. 
One  coordinator  mentioned  that  she  would  use  only  two  or  three  lessons 
on  one^language  topic  instead  of  the  full  10  that  are  available. 

One  coordinator  indicated  an  interest  in  using  the  reading 
curriculum  in  combination  with  language,  so  tbat  her  students*  time 
would  be  evenly  divided  between  math  and  "verbal"  CAI  activitie^L. 

Several  coordinators  said  they  would  have  students  run  their 

own  reports  once  a  week  and  determine  their  own  areas  of  greatest 

need.     Then  each  child  could  select  a  fixed  lesson  in  that  area  for 

extra  drill.    One  coordinator  said  she  would  allow  students  to 

'Choose  extra  work  in  either  the  child's  strongest  or  weakest  area, 

'  so  that  students  would  have  the  opportunity  of  moving  further  ahead 

in  the  areas  in  which  they  excel. 
Motivational  Hints 

Rewarding  students  for  a  high  percent  correct  is  probably  a  less 
effective  method  than  rewarding  for  the  number  of  correct  answers. 
Most  of  the  motivational  hints  listed  earlier  on  page  9  are  appro- 
priate for  students  in  upper  elementary  grades..   Some  examples  of 
teacher-made  awards  are  included  in  the  back  of  this  report.  Also 
included  is  an  equivalence  chart  which  is  helpful  to  students  for 
use  in  the  mathematics  measurement  strand. 
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CAI  OPERATIONS  SECTION 
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CAI:  ^  Job  Description 


The  following  tasks  represent  some  of  the  responsibilities 
borne  by  CAI  Coordinators  (and  ETS)  over  Xhe  last  four  years.  This 
work  must  now  be  shared  by  many  Individuals.  Administrative 
responsibility  for  these  tasks  should  be  established  as  quickly  aa 
possible,  since  much  of  the  work  must  be  accomplished  in  the 
first  month  of  scho^pl. 

First  Month 'Activities  before  CAI  Lab  is  Operational 

.  Getting  the  CAI  lab  upland  running,  including  hardwire 
'maintenance  and  software  preparedness. 

.  Hiring  TAs;  getting  extra  help,  if  needed. 

.  Scheduling/developing  in-service  training  sessions. 

.  Developing  the  CAI  schedule;  promoting  the  CAI  program. 

.  Working  with  teachers  to  develop  student  programs  by 
selecting  CAI  curriculums. 

.  Entering  and  enrolling  students  in  the  selected  CAI 
courses. 

.  Preparing  students  for  CAI  start-up:  readiness 
activities.  ' 


Daily  Operations  in  the  CAI. Lab 

.  Opening  room,  setting  up  equipment. 
.  Cleanitig  terminals  and  maintaining  room  .nvironment. 
.  Monitoring  CAI  schedule;    reporting  prob^»ms. 
.  Assisting  students  at  the  terminals. 
.  Handling  equipment  problems. 
^       .  Providing  motivational  support/aids. 
.  Running  weekly  reports. 

.  Monitoring  student  progress  on.  a  regular  basis. 

.  Enrollitfg  students  in  specific  fixed-strands  or  topics 

;  Maintaining  records/files. 

.  Running  a  back-up;  turning  off  terminals.  ; 
.  Closing  the  CAI  lab.  '  / 
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Coordlnators'  Best  Advice 

Coordinators  were  asked  to  write  down  the  four  or  five  best 
pieces  of  advice  they  could  give  to  people  working  in  the  CAI  lab. 
Their  answers  were  surprisingly  consistent.    They  have  been  combined 
to  produce  the  following  advice: 

•  Have  a  very  organized  schedule  and  stick  to  it. 

Make  no  drastic  changes.    Plan  your  schedule  carefully. 
Once  the  schedule  is  set,  be  firm.     Remember  time-on- 
task  is  important.     (See  Section  on  Scheduling.) 

.  Keep  a  positive  attitude  about  CAI  and  continually 
convey  it  to  students  and  staff  in  the  lab.  Show 

■» 

interest  and  encouragement.     Show  concern  for  student 
projgress  and  maturity.    Be  positive  and  fentKusiastic. 
Remember  the  job  is  routine — keep  your  own  enthusiasm 
up.     Smile  a  lot.     Keep  the' CAI  room  cheerful  and 
seasonal. 

.  Establish  and  follow  through  on  good»  fair  standards 
of  behavior  so  that  no  large  problems  arise.  Maintain 

a 

lab  standards  firmly.    Establish  orderly  traffic 
patterns..   Try  to  monitor  what  students  bring  in — food, 
candy,  gum,  sunflower  seeds!  '  Offer  to  save  it  until 
after  CAI."  Remind  students,  '*We  take  care  of  the 
equipment."    Keep  pupils  on  task  and  out  of  trouble; 
urge  them  to  work  as  many  problems  as  possible. 
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Have  a  lot  of  good,  interesting  motivatj.onal  aids  to  keep 
the  program  alive  and  the  students  doing  well.  Provide 
tangible  goals  for  students  to  work  toward,  and  reward 
their  progress.    Give  positive  verbal  feedback  frequently. 
Develop  some  reward  system  for  good  lessons.  Reward 
students  who  get  a  high  number  of  correct  answers 
rather  than  a  high  percentage  of  correct  answers. 
Motivate  with  pats  on  the  back,  games,  small  prizes. 
(See  samples  of  motivational  aids  at  the  back  of  this 
report. ) 

Keep  good,  open  communication  among  all  participants  in 
the  CAI  lab.     Teachers,  Teaching  Assistants,  and 
Coordinators  are  working  together  for  the  good  of  the 
students.    Be  cooperative.    The  success  of  CAI  is 
related  to  the  importance  teachers  place  on  it.  Train 
TAs  carefully.    Give  teachers  as  much  choice  as 
possible  in  scheduling  and  course  assignment. 

Remember  the  hardware  tasks.     Simple  back-i»ps  and 
clean-ups  help  prevent  larger  problems.    Don*t  worry 
about  the  hardware  tasks.    Relax.    The  equipment  is 
fairly  sturdy,  and  the' damage  you  can  do  is  minimal. 
It  will  not  take  long  to  learn. 


Potential  Problems  with  CAI 


Coordinators  were  asked  what  problems  they  foresaw  with 
running  a  CAI  lab  without  a  full-tiiae  CAI  Coordinator  and  how 
schools  could  best  cope  with  those  problems* 

•  Maintenance  of  the  equipment;..   Teachers  are  intimidated  by 
machines.    Many  more  people  will  be  responsible.    The  phone 
lin^s  between  School  I  and  School  2  have  been  a  problem  for 
two  years;  someone  needs  to  be  present  in  the  School  2  CAf 
lab  to  handle  that  end  of  computer  problems  for  School  I. 

Fantastic,  dedicated  TAs  at  Schools  I  and  2 
c6uld  be  trained  to  handle  equipment  problems. 
Overall,  intensive  in-service  training  for 
both  teachers  and  TAs  is  critical.  All 
teachers  should  be  in-serviced  adequately. 
All  TAs  in  the  schools  should  receive  training. 
Time  will  help  ^s  teachers  begin  to  feel  more 
comfortable.    Student  helpers  might  also  be 
trained  for  some  tasks.    A  chart,  explaining 
the  .JDOst  common^  equipment  breakdowns  and  simple 
solutions,  sjtiould  be  placed  near  the  computer 
idong  with  the  phone  numbers  of  service  repairmen 

•  Maintenance  of  the  CAI  lab:     The  room  environment  may  not 
be  kept  up.    Bulletin  boards  may  not  be  changed  and  visual 
aids  (e.g.,  equivalency  charts)  may  not  be  provided.  The 
terminals  may  not  be  cleaned.    Motivational  aids  could  be 
provided  by  each  classroom  teacher  but,  given  the  human 
factor,  these  may  be  less  than  optimally  successful. 

Solution:     Delineate  responsibilities.    Make  sure  each 
person  knows  what  her/his  responsibility  is. 
TAs  could  carry  most  of  this  responsibility — 
cleaning  terminals,  providing  motivational 
aids,  maintaining  bulletin  boards — especially 


Solution: 
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if  they  used  student  helpers  before  and/or 
after  school. 


The  optimal  use  of  CAI:    The  CAI  lab  may  not  be  used  to 
best  advantage.     Teachers  may  find  It  more  convenient  to 
skip  their  class  time  In  CAI.     The  CAI  lab  may  not  function 
for  a  full  day;  less  than  the  full  day  would  be  a  crime. 
In  small  schools  with  half  the  class  working  at  terminals 
and  half  doing  some  other  assignment,  teachers  will  be 
hard -put  to  give  help  where  help  Is  needed.    No  one  will 
feel  responsible  for  the  total  program.     Who  will  diagnose 
CAI  reports  and  prescribe  assignments? 

Solution:    Cooperation  among  faculty,  students,  voj.unteers, 
and  those  specifically  responsible  for  the  CAI 
lab  will  help.    Delineate  responsibilities. 
TAs  could  be  responsible  for  monitoring 
the  CAI  schedu]^.     CAI  reports  and  their 
Interpretation  could  be  the  joint  responsibility 
of  TAs  and  teachers.     Teach  students  to  run 
their  own  Individual  reports,  diagnose  their 
own  needs,  and  even  assign  themselves  to  work 
In  fixed  strands~lf  possible.  Volunteers 
might  keep  the  lab  open  at  times  when  regular 
personnial  are  not  available — before  or 
after  school. 


Record  keeping;     Who  will  be  responsible  for  keeping  files? 
How  will  student  progress  be  monitored?    Where  will  records, 
be  kept?    I  ^  ^'  ; 

* 

Solutibn:    Prepare  file  folders  for  the  CAI  lab.  Keep 
copies  of  CAI  reports  in  each  class  file. 
*  Reports  at  the  end  of  rapid-motion  are  very 
important  to  help  monitor  student  progress 
during  the  year.    Assign  this  responsibility 
and  establish  the  routine. 


1  ^>  > 
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Example  of  Posted  Directions 

Directions  for  Substitute  in  CAI  lab  - 

1.  Check  to  see  that  alarm  light  (by  door)  shows  "OFF". 

2.  Turn  on  air  conditioner  (smalls  flat»  two  pronged  key-box 
oiKf north  wall  by  breaker  box). 

3*     Turn  on  tenriinals  (switch  on  back»  lower  left). 

Light  will  show  in  30  secpnds.    Press  CR  key  to  get  message: 
"Number  and  name  please." 

'  ^' 

4.  Check  schedule — pass  out  report  sheets  for^ first  class — place 
on  counter  to  left  of  terminal. 

5.  Pass  out  report  sheets  for  next  class  before  first  class 
leaves.    No  assigned  seats — alternate  boys  and  girls. 

6*     Take  roll  for  each  class-^attendance  sheets  are  on  clip  board 
in  file  box  on  first  counter. 

7.  Primary  recess  duty  on  Thursdays  and  Fridays  10:30  to  10:30 — . 
check  with  TA. 

8.  Lunch  time  12:15-1:00.    Leave  keys  in  office  for  TA.    TA  will' 
wipe  screens. 

9.  After  school  TA  does  "Back  up"  on  computer.     If  TA  is  absent 
see  Proct<j)r's  Manual  for  directions.     (Top  drawer  of  filing 
cabinet.): 

10.  Reports  must  be  run  for  teachers  every  Friday.     See  manual  and 
TA. 

11.  All  terminals  must  have  screens  wiped  with  special  cloth 
wipers  stored  under  sink. 

12.  Turn  off  all  terminals. 

13.  Dust  keyboards  with  paint  brush. 

14.  Put  chairs  up  on  counters »  close  blinds»  turn  off  air  conditioner « 

13.     Call  security  (625-6631)  and  tell  them  you  are  going  to  turn 
on  alarm  [  .]    Use  key  marked  S-172  to  activate 

alarm.    You  may  not  go  back  in  lab  again  once  alarm  is  on 
unless  you  call  Security  from  office  first. 


16. 


Return  keys  to  box  in  teachers'  supply  room. 
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Student  Problems  in  CAI 

There  are  probably  fewer  problems  with  students  in  the  CAI  lab 
than  in  most  other  areas  in  school.     The  work  incentive  is  high. 
Nevertheless »  some  problems  exist.     CAI  coordinators  were  asked 
what  kinds  of  problems  they  looked  for  and  how  they  solved  them. 
Some  of  those  problems  associated  with  CAI  lessons  were: 

.  a  student's  pressing  the  wrong  keys 

.  caps  only  —  to  produce  all, capital  letters 

.  control  Z  —  to  end  the  lesson 

.  top  left-hand  keys  —  to  go  off-line 

.  a  student's  inattention  to  CAI  lesson 

\,  .  darkening  the  writing  on  the  CAI  screen  so  that  it  can 
no  longer  be  seen 
.  letting  questions  run  by  without  answering 
•  pressing  the  space  bar  without  making  an  effort  to 
do  the  work 

.  a  student's  frustration 

.  a  student's  interference  with  another's  work 
Speaking  with  children  individually  usually  helps.     Careful  seating 
arrangements  can  solve  some  problems.    Giving  extra  help  to  students 
when  it  is  needed  is  helpful.     Reminding  children  it  is  okay  to 
make  mistakes  helps  to  solve  some  frustration  problems.  One-on-one 
contact  with  the  teacher  or  TA  often  helps.    When  one  child  causes 
another  to  miss  a  problem  by  typing  on  the  keyboard »  subtracting 
points  from  the  offender's  score  may  help.    More  serious  problems 
may  be  handled  by  a  phone  call  or  note  to  the  parents »  or  by  having  a 
child  come  to  the  CAI  lab  with  a  different  class. 


15.3 
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Scheduling  CAI 

Coordinators  were  asked  a  series  of  general  questions  about 
how  they  would  schedule  CAI  In  an  Imaginary  school:  Bursting 
Place.     On  the  next  four  pages  are  listed  those  questions  and  the 
coordinators'  responses. 


Assume  you  are  principal  ^f 
Bursting  Place  Elementary 
School,  and  answer  the 
following  questions; 

How  would  you  assign  CAI,? 

Which  classes  or  grade  levels? 

How  long  a  lab  session? 

How  many  times  a  we^? 

Which  CAI  currlculums? 


Bursting  Place  School:  Hours  9-3 
840  students         32  CAI  Terminals 


Grade 

K 
1 
2 
3 
4 
5 
6 
SDC 


Classes 

6 
5 
4 
4 
4 
3 
3 
3 


Class  Names 

1,  2,  3,  4,  5, 

A,  B,  C,  D,  E, 

F,  G,  H,  I, 

J,  K,  L,  M, 

N,  0,  P,  Q, 

Ri  S,  T, 

U,  V,  W, 

■7,  8,  9 


Independently,  every  coordinator  found  a  way  to  give  CAI  to  every 
student  at  .Bursting  Place  School!     Coordinators  were  so  convinced 
CAI  is  a  worthwhile  activity  that  they  could  not  bear  to  omit  any 
pupils.     Most  coordinators  planned  the  schedule  Xo  give  half  the 
classes  CAI  in  the  first  semester  and  the  rest  of  the  classes  CAI 
in  the  second  semester.     In  general,  all  coordinators  felt  there 
should  be  one  10-minute  session  in  mathematics  for  pupils  in  grades 
1-3  and  two  10-minute  sessions  in  mathematics,  language,  and  possibly 
reading  for  pupils  in  grades  4-6.    They  based  their  rationale  for  one 
or  two  sessions  on  attention  span.    Many  coordinators  felt  that 
students  should  go  to  the  lab  on  a  daily  basis  so  there  is  a 
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regular  routine  for  CAI  drill  and  practice.    An  alternate  schedule 

would  plan  for  pupils  to  go  to  the  CAI  lab  two  or  three  times  a  week, 

The  coordinator  who  planned  the  schedule  below  said: 

The  reason  for  emphasis  on  Grades  4,  5,  and  6  (they 
receive  CAI  both  semesters)  is  that  the  computer  mainly 
helps  individualize  drill  and  practice.    By  grade  4,  the 
'       range  of  levels  in  all  academic  areas  is  great  and  the 
classroom  teacher  needs  more  help  to  individualize  the 
student's  academic  program. 

Semester  1:    Grades  1,  2,  '4,  5>  6,      2  SDC  Classes. 

Men    Tue    Wed    Thur  Fri 


9*10  -    9:20  Class 

A 

B 

A 

B 

A 

9:25  -  9:35 

c 

D 

C 

D 

C 

9*40  -  9:50 

E 

F 

E 

F 

E 

10:20  -  10:30 

G 

H 

G 

H 

G 

10:35  -  10:45 

I 

I 

I 

U:00  -  11:20 

7 

8 

7 

8 

7 

U:20  -  11:40 

N 

0 

N 

0  . 

N 

11:45  -  12:05 

P 

Q 

P 

Q 

P 

Lunch  Break 

1:00  -  1:20 

R 

S 

R 

S 

R 

1:25  -  1:45 

T 

U 

T 

U 

T 

1:50  -  2:10 

V 

W 

'  V 

w 

V 

2:15  -    2:35  MAKE- 

■UPS 

Grades  K,  3,4,  5, 

6,  + 

1  SDC 

Class 

Mon 

Tue 

Wed 

Thur 

Frd 

9:10  -    9:20  Class 

1 

2 

1 

2 

1 

9:25  -  9:35 

3 

4 

3 

4 

3 

9:40  -  9:50 

5 

6 

5 

6 

5 

Recess  Break 

10:20  -  10:30 

J 

K 

J 

K 

J 

10:35  -  10:45 

L 

M 

L 

M 

L 

10:55  -  11:15 

8 

8 

8 

8 

8 

11:20  -  11:40 

0 

N 

0 

N 

0 

11:45  -  12:05 

Q 

P 

Q 

P 

Q 

.  Lunch  Break ^ 

1:00-  1:20 

S 

•R 

S 

R 

S 

1:25  -  1:45 

U 

T 

U 

T 

U 

1:50  -  2:10 

W 

V 

W 

V 

W 

2:15  -    2:35  MAKE- 

-UPS 
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Several  of  the  teachers  who  were  selected  to  participate  in  the  CAI 
program  do  not  wish  their  classes  to  go  to  the  CAI  lab>    Hov  would 
you  handle  this  situation? 

The  responses ,  when  tabulated ,  indicated  two  solutions  had 

equal  weight  with  coordinators  and  could  be  used  in  conjunction 

with  one  another.     One  response  stressed  the  importance  of  positive 

administrative  attitude  and  support  for  all  teachers  to  use  the 

computer*     The  second  response  was  to  involve  the  teacher  by 

demonstrating  the  positive  uses  of  the  CAI  program  for  record 

keeping,  meeting  individual  needs,  and  using  reports  for  aiding 

students*    Administrative  policy  combined  with  friendly  persuasion 

could  be  used  to  encourage  cooperation.     There  was  no  suggestion 

that  the  teacher  or  her/his  class  be  excused  from  CAI. 

Bursting  Place  School  tends  to  track  its  students.     Classes  may 
be  slow,  average,  or  accelerated.     Should  any  changes  in  the 
schedule  be  made  in  light  of  this  information? 

All  coordinators  agreed  that  all  students— including  acceler- 

ated  and  slow  students— should  be  given  CAI,  with  the  emphasis  on 

the  math  curriculum.    The  faster  students  could  be  given  variable 

math  and  possibly  language  or  reading,  while  the  slower  students 

could  be  given  math,  fixed  or  variable,  and  perhaps  no  language  or 

reading  curriculum.     Several  coordinators  made  mention  of  the 

accelerated  learning  and  challenge  that  math  presents  to  the  higher 

achievers.    However,  if  time  and  space  were  very  limited,  some 

coordinators  felt  that  priority  could  be  given  to  the  slower 

students • 
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Burstlng  Place  School  has  been  identified  as  a  low-scoring  school 
in  state-wide  assessment*    The  school  has  developed  a  long-range 
plan  for  upgrading  student  achievement*    How  could  CAI  best  be  used' 
as  part  of  this  3-year  plan? 

Most  coordinators  did  not  change  their  earlier  schedules  oh 

the  basis  of  this  new  information*     One  coordinator  did*     She  felt 

that  by  working  with  students  in  grades  1-4  the  first  year,  1-5  the 

second  year,  and  1-6  the  third  year,  optimal  results  in  test  scores 

would  be  obtained  in  year  3..    Her  plan  was  to  use  variable  strands 

of  math  (and  language  where  it  was  appropriate)  to  identify  weak 

areas  for  individual  students*     Fixed  strands  for  mathematics  (and 

topics  for  language)  could  be  used  once  or  twice  a  week  for  extra 

drill  in  weak  areas.     Cross-age  tutors  could  also  be  provided  as  a 

program  involving  mentally-gifted  minors  at  higher  grade  levels. 

Bef ore-school  and  after-school  programs  on  the  computer  could 

be  run  by  parent  volunteers.     Such  a  long-range  plan  for  the  CAI 

lab  should  help  to  improve  student  achievement  (and  test  scores)  by 

the  end  of  three  years. 

If  a  proposed  new  school  were  built,  it  would  draw  students  away 
from  Bursting  Place  School.     It  would  be  left  with  about  450 
students.    What  CAI  changes  would  you  make? 

Coordinators  almost  unanimously  agreed  that  all  pupils  in 

grades  1-3  should  go  to  the  CAI  lab  for  10  minutes  daily  throughout 

the  school  year  for  the  mathematics  CAI  curriculum;  and  all  students 

in  grades  4-6  should  go  daily,  for  20  minutes,  for  mathematics  CAI 

and  any  other  curriculums  the  teachers  may  recommend  as  needed  by 

the  individual.    This  situation  was  perceived  as  ideal— when  all 

students  could  have  CAI  on  a  routine  basis  all  year. 
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CAL  SP/^£  RACE  WINNERS 


LIGHT  YEARS 


CALrPbWm  GRAND  PRIX 
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Name. 
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k  CUPS  =  1  QUART 


WEIGHT 


16  OZ  =  1  LB 
1000  QM  =  1  KG 


LENGTH 

1  CM 
1  M 
1  KM 
1  FOOT 
1  YARD 


TIME 

60  MINUTES  =  1  HOUR 


10  MM  = 
100  CM  = 
1000  M  = 
12  IN  = 
3  FT  = 
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,  MONEY 


5  PENNIES  =  1  NICKEL 


2  NICKELS  =  1  DIME 


10  PENNIES  =  1  DIME 


COWER-ASSISTED  INSTTOION  AND  C0TOSA7DRY  EDUCATION: 
-  M  Ely  LAUSD  STUDV 

.T>ECAI  a-RRIOJIJUMS 


•This  appeared  as  Part  3  of  M.  Ragosta^^P,  W.  Holland^  and  D,  T,  Jamison 
Computer-Assisted  Instruction  and  Compensatory  Education:  The  ETS/LAUSD  Study. 
Final  Report  to  the  U,  S,  National  Institute  of' Education,  April  1982. 
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^'      Chapter  I 

THE  CAI  tURRICULUMS 

■>      *  .  * 

In.searchTng  for-  CAI  curriculums  for  use  in  this  study,  it  became 
bbvlous  that  there  were  W  reading  or  language  curriculums,  immediately 
available,  which  could  be  offered  to  students  across  grade  levels  l-o. 
Computer  Curriculum  Corporation  (CCC)  could  provljde  Reading,  Grades  3-6 
and  Language  Arts,  Grades  3-6 >  but  lack  of  audio  signals  on  their  systems 
precluded  offering  a  curriculum  for  beginning  readers.    CCC  could  also 
provide  Mathematics  Strands,  Grades  1-6  in  the  same  software  package  as  their 
reading  and  language  programs.     The  decision  was  made ^ to  use  these  off-the- 
shelf  curriculums  for  several  reasons: 

(1)  They  were  CAI  curriculums  Which  were  immediately  available, 
having  had  fairly  wide  usage  in  systems  located  across  the 
United  States.  '  '  ■  * 

(2)  The  CCC  curriculums — especially  the  Mathematics  Strands — had  a  body 
of  evaluation  studies  which,  although  they  had  shortcomings, 

.  appeared  to  indicate  Fuccess  in  raising  test  scores. 

(3)  CCC  was  the  only  vendor  which  could  offer  all  three  curriculums 
in  one  software  package,  at  *a  price  the  study  could  afford. 
Having  one  software  package  assured  the  study  of  a  minimum 
amount  of  time  to  get  the  CAI  operations  running. 

the  CCC  curriculums  are  drill  and  practice  curriculums  which  are  not 
intended  to  teach  students  but  to  reinforce  the  skills  they  have  already 
been  taught  in  the  classroom. 

.   A  brief  description  of  each  of  the  curriculums  follows: 


\ 
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Mathetaatics  Strands,  Gradefe  l-fe" 

Mathematics  Strands,  Grades  1-6  is  one  of  the  most  highly  individual- 
ized CAI  (computer-assisted  instructional)  programs  ever  developed.  A 
student  participating  in  , the  Mathematics  Strands  program  receives  lessons 
that  are  prepared  for  him  oii  the  basis  of  his  own  achievement  and  educa- 
tional needs.    His  lessons'  are  not  stored  in  the  computer's  memory  but 
are  generated  by  the  computer  as  he  works  at  a ^terminal.    Because  the 
computer  immediately  checks  the  student's  response  to  each  item,  it  can 
adjust  the  lesson's  difficulty  level  while  this  lesson  is  in  progress. 
Furthermore,  it. can  make  this  adjustment  in  each  concept  area  on  which 
.the, student  is  working.     Such  individualizing  capability  represents  a 
significant  step  forward  in  the  development  of  curriculum  material  that 
meets  the  goals  of  individualized  instruction. 

All  the  topics  in  elementary  School  mathematics,,  with  the  exception 
of  •  geometry  and  work  problems,,  are  included  in  th^i  Mathematics  Strands 
program.    Its  stress  on  basic  computational  skills  makes  it  compatible 
with  a  wide  range  of  textbook  series.  Because  i^t  does  provide  individual- 
izatlon,  the  program  is  appropriate  for  both  remedial  and  accelerated 
classes*.  .     '  ' 

The  ilathematics  Strands  program  achieves  its  goal  of  individualized 
instruction  by  using  a  strands  structure.    There  are  fourteen  strands, 
one  for  each  concept  area  included  in  the  program.    The  strands  are: 

Strand  1  Number  Concepts 

Strand  2  .                    Horizontal  Addition 

Strand  3  .                     Horizontal  Subtraction 

Strand  4  *                      Vertical  Addition 

Strand  5  Vertical  Subtraction  ^ 

Strand  6  ^  Equations 

Strand  7  ^  Measurement 
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Strand  8 

Strand  9 

Strand  10 

Strand  11 

Strand  12 

Strand  13 

Strand  l4 


Horizontal  Multiplication 

Laws  of  Arithmetic 

Vertical  Multiplication 

Division 

Fractions 

Decimals 

Negative  Numbers 


During  every  session,  the  student  receives  a  mixture  of  items  from 
all  the  strands  that  have  exercises  at  his  grade  level.    The  student's 
work  in  each  of  the  fourteen  strands  is  individuali^d  to  meet  his 
educational  needs.    In  addition,  the  computer  adjusts  the  proportion  of 
exercises  from  each  strand  to  match  the  proportion  of  exercises  covering 
that  concept  in  an  average  textbook. 

Reading,  Grades  3'"6 


Reading,  Grades  3-6  offers  a  supplemental  reading  program  with  two 
important  features:    a  high  degree  of  individualization,  and  a  means  of 
diagnosing  class  and  individual  reading  weaknesses*     Both  these  features 
make  the  program  a  useful  tool  in  building  toward  the  goal  of  individual" 
ized  instruction. 

The  program  Consists  of  practice  items  designed  to  sharpen  the 


student's  reading  skills  in  five  areas:    word  analysis,  vocabulary 
extension,  comprehension  of  sentence  structure,  interpretation  of  written 
material,  and  development  of  study  skills.     It  contains  enough  material 
for  four  years  of  work  at  grade  levels  3,  4,  5,  and  6,  as  well  as  supple- 
mentary remedial  material  that  extends  to  grade  level  2.5. 


Each  student  moves  /through  the  program  at  his  own  pace.    The  difficulty 


c 
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student  needs  remedial  work,  the  program  moves  him  to  a  lower  grade 

0 

level.     If  a  student  needs  to  advance  to  more  challenging  material,  the 
program  moves  him  forward  rapidly.    The  students  in  a  given  class  may 
spread  in  grade  placement  over  every  grade  year  the  program  covers. 

The  program  uses  a  strands  structure  to ^individualize  each  student's 
lessons.  Each  of  the  five  skill  areas  the  program  includes  is  represented 
by  a  strand,  or  graduated  sequence  of  related  items.    The  strands  are: 

Strand  A  Word  Attack 

Strand  B  Vocabulary 

Strand  C  Literal  Comprehension 

Strand  D  Intetpretive  Comprehension 

Strand  E  Work-Study  Skills 

If  the  student  is  doing  very  well  in  one  area,  he  moves  forward 
rapidly  in  the  strand  which  contains  items  from  that^area.    If  he 'is 
performing  poorly  in  one  area,  he  is  moved  back  in  that  strand  until^^he 
reaches  a  level  that  is  suited  to  his  abilities.    At  this  point  he  can 
begin  to  move  forward  again.    His  rate  of  movement  in  each  strand  is  not 
affected  by  his  position  or  rate  of  movement  in  the  other  strands. 

The  program  was  designed  with  low  reading  levels  in  mind.     It  begins 
with  very  simple  vocabulary  and  adds  words  from  carefully  selected 
vocabulary  lists.    The  lists  concentrate  on  words  that  children  encounter 
in  reading  materials  and  daily  life  situations. 

A  special  section  at  the  lowest  levels  of  the  program  contains  basic 
two-  to  five-word  sentences.     It  is  included  as  remedial  work  for  those 
students  in  any  school  grade  who  may  need  it.    The  material  in  this 
section  is  not  differentiated  into  strands. 
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Reading  for  Comprehension 

The  Reading  for  Comprehension  curriculum  is  a  newer  version  of 
the  reading  curriculum  which  has  already  been  described.    The  section  on 
basic  sentences  has  been  removed  and  a  new  paragraph  strand  added.  Each 
grade  level  of  Reading  for  Comprehension  contains  100  paragraphs  with 
associated  questions. 

Reading  for  Comprehension  consists  of  the  following  6  strands: 

A.  Word  Attack       analyzing  words  as  units 

B.  Vocabulary  ~  building  a  reading  vocabulary; 

C.  Literal  Comprehension  —  understanding  the 
literal  meaning  of  sentences 

Dm  Interpretive  Comprehension  ~  learning  to  read 
between  the  lines;  developing  critical  reading 
skills 

E.  Word-Study  Skills  ~  learning  to  alphabetize  " 
and  to  use  resources  effectively 

P.    Paragraphs        integrating  skills  to  read  an  entire 
paragraph  and  answer  related  questions* 

Language  Arts,  Grades  3-6 

Language  Arts,  Grades  3-6  attacks  today's  most  common  language  usage 
problems.  It  covers  grades  three  through  six  with  enough  material  foir  a 
year's  work  at  each  grade  level.     In  addition,  it  offers  supplementary 
material  designed  for  students  with  special  langufi^ge  problems:  hearing- 
impaired  students  and  students  for  whom  English  is  a  second  language. 

Lai>guage  Arts,  Grades  3-6  supplements  almost  any  language  arts 

textbook  or  teaching  method.     It  stresses  udage  instead  of  grammar  and 

t 

uses  very  few  grammatical  terms.     Students  using  the  program  learn  by 

/' 

example,  pattern,  and  practice. 

1 
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The  program  has  two  sections,  strands  and  topics.     In  the  first  year 
of  this  study,  only  the  strands  section  was  used*    The  strands  section 
supplies  individualized  drill  and  practice  tailored  to  each  student's 
achievement  level.     In  the  topics  section,  students  receive  lessons  on  a 
topic  assigned  by  their  teacher. 

The  strands  section  consists  of  eight  strands,  or  strings  of  items* 
Each  strand  covers  grades  three  through  six.     The  program  keeps  records 
of  each  student *s  performance  in  every  strand  and  uses  this  information 
to  adjust  the  student's  level  in  each  strand.     The  strands  are: 


Strand  A  Principal  Parts 

Strand  B  Verb  Usage 

Strand  C  Subject-Verb  Agreement 

Strand  D  Pronoun  Usage 

Strand  £  Contractions,  Possessives,  and  Negatives 

Strand  F  Modifiers 

Strand  G  Sentence  Structure 


The  Curriculums  in  the  Study 

The-  Mathematics  CAI  curriculum  was  used  at  all  grade  levels  of 
elementary  school*     Kindergarten  students  were  not  a  part  of  the  study 
but  ddd  use  the  CAI  lab  in  one  school  on  a  regular  basis*     First  gi;a^ers 
quickly  learned  to  use  the  equipment  and  were  challenged  by  the  Mathematics 
curriculum*     Only  two  students  in  the  4  years  of  the  study  topped  out  of 
mathematics  strands  curriculum:    both  were  girls  and  both  had  received  one  or 
two  sessions  of  mathematics  daily  across  4  grade  levels* 

The  Language  curriculum  was  used  in  grades  3-6*     A  few  third  graders 
were  non-readers  or  Spanish-speaking  and^  could  not  understand  or  benefit 
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from  the  Language  curriculum.    Many  sixth  graders  topped  out  of  the 
Language  curriculum. 

The  Reading  curriculum  was  used  in  grades  4-6.     It  was  even  harder 
for  non-readers  or  Hispanic  children  to  access  even  at  grade  4,  but  fewer 
sixth  graders  topped  out  of  reading  than  out  of  language.    The  newer 
Reading  for  Comprehension  curriculum  was  used  only  in  grade  4  in  the 
final  year  of  the  study.     It  also  proved  difficult  for  students  of  low 
reading  ability. 

Summary 

In  this  chapter  we  have  briefly  described  the  4  CAI  curriculums 
used  in  this  study.     For  further  information  and  descriptions  of  the  CAI 
curriculums,  the  Computer  Curriculum  Corporation  of  Palo  Alto,  California, 
may  be  contacted. 


,  1 
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Chapter  II 


TIME  ON  THE  COMPUTER 


In  this  chapter  we  will  examine  the  amount  of  exp&sure  to  CAI 
experienced  by  students  in  the  study;  we  will  examine  the  question  of 
whether  students  assigned  to  high  or  low  levels  of  a  curriculum  actually 
received  high  or  low  levels  of  CAI  experience  in  that  curriculiim;  and  we 
will  examine  whether  the  amount  of  CAI  time  differed  with  studejnt 
characteristics  such  as  sex  or  ethnicity. 

We  have  selected  specific  data  bases  for  analysis.    The  grade  4, 
year  4  data  was  selected  because  it  contained  the  largest  data  base  for 
any  single  year  and  grade  level.    The  three  longitudinal  data  biases  were 
also  selected  because  of  their  importance  to  the  study. 


Total  Time  on  the  Computer  y 

Each  year  except  for  the  fil^st  year  of  the  study,  students  started 
work  it.  the  CAI  lab  about  the  middle  of  October  and  ended  about  the 
middle  vf  June.    During  that  p^eriod  of,, time — barring  exceptions — students 
assigned  to  one  session  of  CAI  daily  received  50  minutes  of  CAI  each  week, 
while  students  assigned  two^seusions  received  100  minutes  weekly.  There 
were  exceptions,  however./  There  were  holidays,  early  closings ,  assemblies, 
school  trips,  fire  drills,  and  compi/ter  breakdowns  as  well  as  student 
absences.    Two  questions  concern  us.    How  much  variability  exists  among 

7  / 

students  assigned  a  specific  amount  o^  CAI?    How  much  CAI  did  the  average 
student  receive? 
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Grade  4,  Year  4.    Students  in  grade  4  in  the  final  year  of  the  study 
were  assigned  one  10-minute  session  of  CAI  daily  in  1  of  4  CAI  curriculums: 
mathematics,  reading,  language,  or  reading  for  comprehension.    The  boxplot 
shovm  in  Figure  1  indicates  that  students  in  each  of  the  four  curriculums 
received  about  900  minutes  (or  90  sessions)  of  CAI  during  the  school  year. 
Fifty  percent  of  the  students  in  each  curriculum — those  between  the  lowest 
quartlle  and  the  upper  quartile-Treceived  between  80  and  100  sessions  of  CAI 
during  the  year.     Some  students  received  less  than  500  minutes  of  CAI  while 
others  received  up  to  1300  minutes.    Despite  the  wide  range  of  individual 
differences  in  exposure  to  CAI,  the  differences  across  treatments  are  so 
minimal  as  to  be  practically  nonexistent. 
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Grades  1-3.     In  this  longitudinal  CAI  cohort,  students  were  randomly 
assigned  by  classrooms  to  mathematics  CAI  during  grades  1  and  2.     In  grade  3, 
students  were  randomly  assigned  within  classrooms  to  mathematics  or  language 
CAI.     Over  the  three  years,  some  students  received  CAI  for  only  one  year  while 
others  received  CAI  for  two  or  three  years  or  not  at  all.     Our  concern  with 
this  cohort  is  whether,  in  fact,  students  received  two  or  three  times  the 
one-year  treatment  levels  if  they  were  assigned  to  CAI  for  two  or  three  years. 
In  Figure  2,  we  see  that  students  who  received  two  or  three  years  of  mathematics 
CAI  did,  in  fact,  get  slightly  more  than  two  or  three  times  the  amount  of 
CAI — on"' the  average— as  the  one'-year  group.    One-year  students  averaged  93 
sessions;  two-year  students  averaged  197  sessions;  and  three-year  students 
averaged  288  sessions. 
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Grades  3-5 «     Students  in  this  longitudinal  CAI  cohort  were  exposed 
to  one  of  two  treatment  conditions.     Students  who  were  randomly  assigned  to 
two  sessions  of  mathematics  CAI  (MM)  in  grade  3  continued  with  that  treatment 
through  grade  5.     Other  students »  who  had  been  randomly  assigned  to  one 
session  of  mathematics  and  one  session  of  language  arts  in  grade  2  (ML), 
received  one  session  of  reading  and  one  of  language  (RL)  in  both  fourth  and 
fifth  grade.    Our  concern  with  this  CAI  cohort  is  whether  the  MM  group 
and  the  ML/RL  group  received  equal  amounts  of  time  on  the  computer  over 
the  three  years.     Figure  3  shows  almost  identical  amounts  of  time  for  the  two 
groups:    each  group  received  on  the  average  slightly  more  than  500  10-minute 
sessions  on  the  computer. 
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Grades  4-6.     Students  in  this  longitudinal  CAI  cohort  received  s»two 
lO-minute  sessions  of  CAI  daily  for  three  years.     They  were  randomly  assigned 
to  mathematics  (MM),  reading  and  language  (RL),  or  mathematics  and  alternating 
sessions  of  reading  and  language  (MRL).     Figure  4  indicates  the  relative 
amounts  of  time  on  the  computer  for  eabh  of  the  three  treatment  conditions. 
Although  the  RL  gr6up  received  less  CAI  time,  only  25  or  30  sessions  (12-15 
days)  of  CAI  separate  the  RL  group  from  the  others*    We  will  take  a  closer 
look  at  this  CAI  cohort  to  determine  whether  differences  in  the  levels  of 
treatment  threaten  the  research  design. 
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Levels  of  CAI  Treatment 

The  research  design  provided  for  an  MRL  group  in  the  grades  4-6  CAI 
cohort  because  the  MRL  group  represented  an  intermediate  treatment  condition 
between  the  MM  group  and  the  RL  group.    With  regard  to  mathematics,  for 
instance,  the 'RL  group  received  zero  sessions,  the  MRL  group  receiver'  one' 
session  daH.y,  and  the  MM  group  received  two  sessions  daily*     It  was  important 
to  the  research  desiign  that  the  MRL  group  receive  about  half  the  ^CAI  time  in 
the  curriculums  as  other  groups.    In  Figure  5  we  see  tliat  those  conditions  ^ 
were  met.    In  mathematics  the  MM  group  took  an  average  of  493  sessions  while 
the  MRL  group  took  248.    The  RL  group  took  239  sessions  of  reading  while  the 
MRL  group  took  119.     In  langua,.ge  CAI,  the  RL  group  took  222  sessions  while  the 
MRL  group  took  118. 

The  language  curriculum  was  easier  to  complete  than  the  other 
CAI  curriculums  in  this  study.    More  students  topped  out  of  language  and 
were  unable  to  accumulate  additional  time  in  that  curriculum.    That  fact 
could  help  to  explain  the  slightly  depressed  score  for  the  RL  group  in 
language.    If  the  RL  group  had  had  235  or  240  sessions,  the  groups  would 
have  been  more  nearly  equivalent.    However;  students  who  topped  out  of 
language  were,  in  fact,  enrolled  in  language  topics'^^a  variation  of  the 
CAI  language  curriculum.    Records  were  not  kept  of  how  much  time  was  ^ 
spent  in  tooics,  but  that  treatment  was  used  to  maintain  the  conditions 

r 

of  the  research  design. 

Since  the  conditions  of  the  research  design  were  met  and  students 
assigned  to  different  levels  of  CAI  treatment  acuially  obtained  those 
different  levels,  the  results  of  the  effectiveness  studies  have  not  been 

0 

compromised.    We  now  turn  our  attention  to  whether  the  amount  of  CAI  time 
differed  with  the  sex  or  ethnicity  of  CAI  students. 
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CAI  Time  by  Sex 

Time  breakdown  by  treatment  and  sex  for  students  in  grade  4,  year 
4,  shows  that,  at  most»  only  five  sessions  separates  the  means  of  males  and 
females  in  any  of  the  four  curriculum  assignments*    The  greatest,  difference 
occurs  for  the  language  curriculum,  and  topping  out  may  have  affected 
those  means  slightly.    When  boxplots  were  run  on  the  time  breakdown  for 
longitudinal  CAI  cohorts  no  discernible  sex  pattern  emerged*  , 

CAI  Time  by  "Ethnicity 

Time  on  the  computer  by  treatment  and  ethnicity  for  mathematics, 
reading  and  language  CAI  students  in  grade  4^  year  4,  is  presented  in 
Figure  6;    A  disturbing  pattern  emerges  showing  lesser  amounts  of  CAI 
time  for  Black  students  than  for  Hispanics  or  others.    Th^t  pattern 
frequently  repeats  itself  for  students  in  the  longitudinal  CAI  cohorts. 
In  grades  4-6  the  Hispanic  students  received  less  time  in  the  MM  and  MRL 
conditions  than  did  Black  students  but,  in  general.  Blacks  obtained  the 
lowest  times,  Hispanics  the  median  amounts  of  time,  and  other  students 
the  most  time.    Since  ethnicity  and  schools  were  confounded  to  a  large 
extent  in  this  study,  further  analyses  were  done  plotting  treatment  by 
school.    Scfibol  3,  which  was  predominantly  Black,  had  the  lowest  CAI 
time;  School  4,  whiwh  had  the  largest  Hispanic  population,  was  next;  and 
School  2,  whose  population  was  to  a  large  extent  White,  had  the  highest 
CAI  times.  ^- 

Since  the  research  design  called  for  random  assignment  of  students 

.  •    *  \    ' . 

to  curriculums  within  schools,  e^t/hnicity  should  be  randomly  distributed 
across  treatment  conditions,.    Nevertheless,  both  sex  and  ethnicity 
variables  have  been  us^d  as  covariates  in  all  regression  analyses  to  be 
reported  in  this  study. 
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Figure  .6.    Time  Breakdown  by  Treatment  and  Ethnicity:    Grade  A,  Year  4. 
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Summary 

In  this  chapter       have  examined  the  amount  of  exposure  to  CAI 
experienced  by  stude/ts^n  the  study,    Althou^  there  was  great  variability 
in  the. total  amounts  of  CAI  time  for  individual  students,  the  overall 
differences  across  treatment  groups  were  minimal*      Where  students  were/ 
assigned  different  levels  of  CAI  exposure  in  the  study  they  received 
different  levels  of  exposure.    Although  no  consistent  jdif feridnces  in  CAI 


time  were  found  between  males  andv^females,  differenced  we^  found  between 

s\ 

minority  students  and  others.    Lower  CAI  times  for  Black  students  were 

7  i    \  •  / 

/  \  /  '  ■  . 

related  to  lower  CAI  times  in  one  predominantly  Bl4ck  school. 


Chapter  III 
CURRICULUI4  PLACE^IEJ^IT 

The  CAI  curriculums  ^ve  been  described  as  adapting  to  the  ability  / 
levels  of  various  students.     In  this  chapter  we  will  take  a  closer  look 
at  those  claims.     We  will  examine  the  rapid  motion  which  is  in  effect  ^ 
during  the  first  10  sessions  of  a  CAI  curriculum;  we  will  take  a  close 
look  at  the  initial  curriculum  placement  for  students  in  grade  4,  Year  4; 
and  we  will  take  an  arbitrary  entry  level  for  the  mathematics  curriculum 
and  see  what  kinds  of  students  at  what  grade  levels  are  represented  by 
that  starting  point. 

Rapid  Motion 

The  CAI  curriculums  used  in  this  study  can  adjust  each  student's 
grade  level  up  or  down  in  half-year  steps.     This  rapid  adjustment  of  ; 
grade  level  is  called  initial  placement  motion.     It  comes  into  effect 
only  during  the  first  10  sessions  and,  only  if  a  student  performs  very 
poorly  or  very  well  at  the  entering  grade  level. 

Students  who  answer  50%  or  fewer  of  the  items  correctly  during  any 
of  the  first  ten  sessions  move  back  a  half  year:     for  example,  from  an 
entering  grade  level  of  4.0  to  a  new  grade  level  of  3.5.     Students  who 
answer  95%  or  more  of  the  items  correctly  move  forward  a  half  year:  for 
example,  from  grade  level  4.  to  grade  level  4.5. 

The  initial  placement  motion  is  one  of  the  individualizing  elements 
of  the  program.     It  ensures  that  students  work  at  the  level  appropriate 
to  their  ability. 

To  illustrate  rapid  motion  we  asked  Jezebel — a  fifth-grade  student, 
one  of  the  only  two  students  who  topped  out  of  the  mathematics  CAI  curriculum 
to  try  to  top  out  of  mathematics  within  10  sessions  starting  from  three 
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Table  3-1 

The  Rapid  Motion  Progress  of  Jezebel  from  Three 
'Entry-Level  Placements. 
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different  entry  levels.    Table  3-1  indicates  Jezebel's  progress  and 
placement  on  her  three  attempts.    As  student  //801  she  entered  the  curricu- 
lum at  level  6.0  and  after  session  //lO  she  had  a  mean  curriculum  placement 
of  7.0,  with  strand  placement  ranging  from  6.6  to  7.4.    Only  twice  did 
she  get  more  than  95%  of  her  questions  correct  while  she  averaged  53 
attempts  each  session. 

On  her  second  try— beginning  from  a  curriculum-placement  of  5.0— 
Jezebel  did  better.     On  four  occasions  in  her  first  10  sessions  she  got  95 
percent  or  more  of  the  questions  correct  and  her  final  placement  (at  the 
end  of  11  sessions)  was  7.2.     An  interesting  phenomenon  is  illustrated  in 
Jezebel's  second  attempt.     After  she  had  worked  for  20  minutes  on  day  2 
as  student  y/802,  she  signed  on  for  a  third  session  of  mathematics.  She 
immediately  realized  she  should  only  have  done  two  sessions,  and  she  stopped 
her  lesson  before  she  had  ainswered  a  question.    The  computer  counted  it 
as  session  y/5  and  scored  her  zero.     Despite  that,  Jezebel  made  a  Jump  of 
2.2  curriculum  levels  in  her  first  hour  and  forty  minutes.     When  the  nine 
meaningful  sessions  were  averaged,  Jezebel  had  attempted  a  mean  of  53 
questions  per  session — the  same  number  of  attempts  as  in  her  first  round 
of  sessions. 

» 

As  student  y/803  Jezebel  attempted  to  top  out  in  10  sessions  starting 

from  level  4.0  of  the  mathematics  CAI  curriculum.     Only  once  in  this  set 

1 

of  records  did  she  reach  95%  or  better.     The  mean  number  of  questions 
attempted  on  this  try  reached  more  than  88.     Her  final  placement  was  a 
discouraging  5.0. 


1 


-21- 


One  can  see  from  Jezebel's  reports  that  rapid  motion  does  Work* 
If  high  percentages  of  correct  scores  are  obtained,  the  movement  Is 
rapid.    It  Is  unfortunate,  perhaps,  that  the  first  10  sessions  of  a  CAI 
curriculum  come  at  a  time  when  students  find  It  easy  to  make  errors 
because  of  unfamlllarlty  with  the  system.    Errors  are  easy  to  make  even 
for  an  expert  with  the  system  like  Jezebel. 

Fall  CAI  Placement;    Grade  4 

Students  entering  grade  4  In  Year  4  had  never  been  exposed  to 
computer-assisted  instruction  although  many  of  them  had  taken  the  curriculum- 
specific  tests  (CSTs)  developed  from  the  CAI  currlculums.    Figure  7 
gives  a  good  Indication  of  the  variability  of  entry-level  placement  after 
rapid  motion. 

Differences  among  currlculums.    Of  the  three  CAI  currlculums  In 
general  use  In  the  study-Tnnathematlcs,  reading  and  lanaguage  —conventional 
wisdom  was  that  language  was  the  easiest  and  mathematics  was  most  difficult. 
There  were  several  kinds  of  observations  to  this  effect.    During  the 
first  year  of  the  study,  sixth-grade  students  topped  out  of  language  more 
frequently  than  out  of  reading.     Fourth-grade  students  had  less  difficulty 
with  language.    The  100%  Club  more  frequently  acquired  members  from 
groups  assigned  to  reading  and  language  than  to  mathematics.    Figure  7 
may  be  giving  another  bit  of  evidence  for  the  conventional  wisdom.  Since 
students  were  randomly  assigned  to  currlculums,  one  would  expect  the  mean 
entry  levels  of  equally  difficult  currlculums  to  be  approximately  equal. 
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Figure    7.     Fall  Placement  for  Schools:    All  Currlculums:     Grades  4 
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Entry  levels  range  from  a  low  mean  *of  3.8  in  mathematics,  through  meaps 
of  4,3  and  4.5  in  the  two  reading  curriculums,  to  a  high  mean  of  5.1  in 
the  language  curriculum* 

.,Dif f erences  among  students >    The  wide  range  of  ability  levels  of 
students  in  grade  4  in  the  CAI  schools  is  apparent  from  the  wide  range  of 
entry  levels  in  each  of  the  four  CAI  curriculums  seen  in  Figure  7.  In 
mathematics,  students  began  their  drill-and-practice  program  as  low  as 
level  1.5  or  as  high  as  6.3.    Minimums  in  reading,  language  and  comprehen- 
sion were  2.5,  3.2,  and  3. 2,  respectively,  while  maximums  were  6.8,  7.0, 
and  7.0.     The  low  of  2.5  in" reading  and  the  highs  of  7.0  in  language  and 
comprehension  represent  end  points  in  those  curriculums.    Even  at  grade  4 
some  students  could  barely  access  the  reading  curriculum.  Non-readers 
and  non-English  speaking  children  had  difficulty  with  the  language 
curriculum  as  well  as  the  reading  and  comprehension  curriculums.  Other 
grade  4  students  reached  the  upper  limit  of  these  curriculums  and 
topped  out  within  the  first  10  sessions.    The  fact  that  some  students 
had  difficulty  accessing  the  curriculums  while  other  students  topped  out 
is  a  severe  limitation  of  the  reading  and  language  CAI  curriculums. 
Although  they  appear  to  span  several  grade  levels,  they  were  not  broad 
enough  to  accommodate  the  range  of  skills  of  students  in  grade  4  in  our 
CAI  schools.    The  mathematics  curriculum,  on  the  other  hand,  could 
accommodate  students  from  kindergarten  to  grade  6  with  no  difficulty. 
/  The  only  two  students  to  top  out  of  the  mathematics  curriculum  had  had  20 

minutes  of  mathematics  CAI  daily  for  almost  four  years  and  were  both 
extremely  bright  girls. 
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CAI' Placement;    The  Currlculums 

Another  way  to  look  at  the  adaptability  of  the  curriculums  to  the 
ability  levels  of  students  is  to  take  a  specific  range  of  curriculum 
levels  and  determine  how  many  students  at  each  grade  level  have  used  that 
entry  level.    Table  3-2  presents  that  data  for  the  students  in  the  CAI 
study.     The  variation  in  ability  levels  of  students  is  demonstrated  by 
the  capability  of  the  CAI  curriculums  to  range  widely  across  entry  levels 
even  within  a  specific  grade  level.     Students  in  the  mathematics  CAI 
curriculum,  for  example,  ranged  from  a  low  entry  level  of  1.5-1.9  through 
the  middle  ranges  to  a  high  entry  level  of  6. 5-6. 9. 

The  breadth  of  the  mathematics  CAI  curriculum  is  also  demonstrated 
by  Table  3-2.    Students  entered  the  curriculum  at  all  grades  and  had 
plenty  of  room  to  grow.     The  reading  and  language  programs  are  more 
restricted,  serving  fewer  grades  at  the  lower  end  of  the  curriculum  and 
providing  less  extensive  coverage  at  the  upper  end.     Perhaps  one  of  the 
reasons  for  the  difficulty  in  assessing  the  effectiveness  of  thao  reading 
and  language  curriculums  is  the  fact  that  the  range  of  ability  levels 
among  students  was  greater  than  the  capacity  of  the  current  CAI  curriculums 
in  reading  and  language. 
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Table  3-2 


(Ambers  of  Students  by  Actual  Grade  Levels,  Entering 
Each  of  3  CAI  Currlculums  at  Varying  Entry  Levels 
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Summary 

In  this  chapter  we  have  looked  at  the  capability  of  the  CAI  curriculums 
to  adapt  to  the  ability  of  students.    We  have  examined  the  rapid-Tnotion 
phase  of  the  curriculums  in  effect  during  the  first  10  sessions.    We  have 
taken  a  close  look  at  the  placeinent  of  fourth  graders  and  a  broad  look  at 
the  placement  of  all  CAI  students  in  the  study.    We  turn  our  attention 
now  to  the  progress  made  by  students  in  the  CAI  curriculums. 
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-       '  Chapter  IV 

PROGRESS  IN  CAI  CURRICULUMS 

Coordinators  in  the  CAI  labs  reported  on  their  perceptions  of 
progress  in  the  GAI  curriculums.    As  ^background  information  their  percep- 
tions  may  be  of  interest.    Coordinators^  strongly  approved  the  mathematics 
curriculxim  and  felt  that  student  progress  in  that  curriculum  was  appropriate 
and  m6tivating  to  students.    The  language  curriculum  was  also  motivating 
and  student  progress  in  language  arts  was  relatively  fast.'  Student 
progress  in  the  reading  curriculum,  however,  was  slow  and  methodical.  A 
few  highly  motivated  students  complained  at  the  lack  of  movement,  and 
coordinators  forwarded  those  complaints  to  project  personnel. 

Most  students  in  grades  4-6  were  V:ery  interested  in  their  own  daily 
success  rate  and  in  their  weelkly  progress  in  CAI.     In  a  drill-and-practice 
curriculum  which  varies  little  over  time,  a  student's  progress  in  that 
curriculum  is  a  strong  motivational  tool. 

In  this  chapter  we  will  take  a  close  look  at  student  progress  in  the 
CAI  curriculums;  we  will j discuss  rate  of  progress  by  grade  and  entry 
level  for-all  CAI  students^in  the  study;  and  we  will  discuss  differences 
among  the  curriculums.  in  the  light  of  student  progress. 
Student  Progress 

Figures  8  and  9  are  graphs  of  student  CAI  placement  against 
CAI  time  and  give  a  visual  indication  of  progress  in  mathematics,  reading 
and  language. 
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The  mathematics  curriculum^     Figure  8  contains  six  plots  of  the 
mathematics  curriculum,  one  plot  at  each  of  the  six  grade  levels.  The 
plots  for  grades  2,  3,  4  and  5  are  especially  interesting  because  they  . 
show  two  levels  of  assigned  CAI  treatments  in  mathematics.     In  grade  2, 
year  1,  students  within  classrooms  were  assigned  either  7  (M)  or  14  (MM) 
minutes  of  mathematics  CAI  daily.     The  graph  shows  the  students  progressing 
at  about  the  same  rate  with  the  MM  group  getting  a  little  further  along 
in  the  curriculum.     In  grade  3  the  groups  were  reversed,  with  most  MM 
students  assigned  to  mathematics  and  language  (ML)  rather  than  two 
sessions  of  mathematics.     Rates  of  progress  for  the  two  groups  are 
similar  but  the  MM  group  went  further  into  the  curriculum.  Rather 
similar  results  are  apparent  in. grades  4  and  5.     Students  assigned  to 
double  sessions  of  mathematics  (MM)  in  grade  4,  year  1,  progressed 
further  in  the  curriculum  than  students  assigned  to  mathematics  with 
alternating  sessions  of  reading  and  language  (MRL).    When  students 
maintained  their  CAI  treatments  in  grade  5,  ,the  rate  of  progress  for  the 
MM  group  appears  to  be  widening  the  differences  in  curriculum  placement. 
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Figure  8.    CAI  Progress  in  Mathematics 
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The  reading  and  language  currlculums>     In  Figure  9  in  the  reading 
curriculum  we  see  a  different  pattern.    In  grades  4  and  5,  students 
assigned  to  CAI  reading  and  language  curriculums  (RL)  appear  to  be 
progressing  at  equal  rates  on  parallel  paths  with  the  MRL  group »  except 
that  the  RL  group  is  further  into  the  curriculum  almost  from  the  beginning^ 
It  looks  as  though  rapid  motion  may  have  determined  the  placement  of  the 
two  groups  differently,  but  after  completing  the  100  minutes  of  rapid  motion, 
students  in  both  groups  proceeded  at  approximately  equal  rates*  Why 
would  rapid  motion  affect  the  two  groups,  so  differently?    Perhaps  it  is 
because  the  two  groups  are  less  similar  than  the    design  intended.  In 
fact,  non*-readers  and  predominantly  Spanish^speaking  children  were 
removed  from  the  RL  group  when  it  was  determined  that  the  reading  and 
language  programs  were  inappropriate. 

In  Figure  9,  the  progress  of  the  same  RL  and  MRL  students  is 
portrayed  graphically  for  the  language  curriculum  also.     The  graphs 
seem  to  show  the  RL  students  accelerating  faster  in  language  than  the  MRL 
students.     Perhaps  one  session  of  language  every  other  day  is  tod  little  to 
achieve  the  maximum  rate  of  progress. 


207 


-31- 


R 

0 
I 

P 
L35 

C 

«30 
E 
N 

20 


60 

R 

0 
I 

p 

L45 

A 
C 

^40 
E 
N 

T35 


30 


ft 

|w 

0 

V 

N50 

p 

L49 

A 
C 

^40 

E 
N 

T35 


30 


CAI 

-  GK4,  TU 

.  CCC  tlADIKG  -  IL(0)  4  MXL(X)  - 

MEANS 

1 

1                   1  i 

1 

0 

0 

0  0 

U 

X  X 

K 

— 

1         1  1 

1 

) 

Attn  onn 

rcAN  rtic  OH  cohputer 

CAI  - 

CCC  READING  -  l«L(0>  i  RL<K) 

-  MEANS 

1 

1                I  I 

1 

X 

K 

0  0  0 

n  n 

0  0  °  ° 

X      X  - 

1 

1         1  1 

1 

CAI  -  GRA,  TXI  -  CCC  LANGUAGE  *  RL(C)  4  MRL(X)  -  MEANS 


200 


400  SOO 
REAN  rtrC  ON  COflPUTER 


800 


1000 


CAI  -  CR6»yRi  -  CCC  REAOINC  •  RL(0>  t  RR(^>  -  nEANS 


"I  1  1  ]  1  1  r 


0  ♦ 


400  too 

riEAM  TIME  OH  COTIPUTEP 


1200 


400  BOO 

fCAN  T I  HE  ON  COfTUTEP 


ERIC 


Figure  9.    CAI  Progress  in  Reading  and  Language  Arts. 
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Rate  of  Progress 

Norm  tables  for  students  in  this  CAI  study  were  made  to  display 
entry  and  exit  levels »  mean  time  on  computer  and  rate  of  progress.  The 
norm  tables  were  further  broken  down  by  sex  and  ethnicity  within  grade 
levels.    An  ex£unple  of  the  norm  table  for  mathematics  entry  level  1.5  to 
1.9  is  shown  in  Table  3-3. 

_Rate  of  progress  as  shown  in  the  norm  table  is  defined  as  curriculum 
levels  attained  per  100  minutes  of  CAI  time.     It  was  computed  for  each 
student  by: 

exit  level  ~  entry  level  jqq 
exit  time  -  entry  time 

In  Table  3-3  we  see  that  the  mean  rate  at  this  entry  level  for  all 

students  in  grade  1  was  .075.     With  roughly  1000  minutes  of  CAI. time,  the 

average  first  grader  could  cover  .75  of  a  curriculum  level.    In  fact,  the 

average  first  grader  at  this  entry  level  entered  at  1.66  and  left  at  2.45 

with  1041  minutes  of  CAI  time,    that  information  was  obtained  from  the 

bottom  line  of  the  entries  for  grade  1.    Those  figures  are  articulated 

•even  further  with  breakdown  by  sex  and  ethnicity:    Black,  Spanish, 

and  others. 

In  looking  across  grade  levels,  the  rate  of  progress  for  students 
entering  the  math  curriculum  at  level  1.5-1.9  increases  to  grade  4  and 
then  decreases  in  grade  5.     (The  increase  in  grade  6  is  based  on  one 
student  and  should  not  be  considered  in  looking  for  patterns  in  the 
data.)    The  higher  the  rate  of  progress  figures,  the  farther  the  student 
has  been'  able  to  proceed  in  the  curriculum.     The  mean  exit  scores  do  not 
show  the  pattern  of  increase  through  grade  4  because  of  the  different 
amounts  of  CAI  time  at  the  different  grade  levels. 


Table  3-3 

Norm  Tables  for  CAI  Achievement  (LAUSD/CAI) 


SruOENTSI  H 


ENTRr 
NEAN 


FOU  NATH  ENTRV  LevEL  1.5-1.9 
EXIT 


GRADE  II 

OTHERS   I  2T 

BLACK     I  ,5. 

S»'AMI$M|  n. 

MALE       I  28. 

FEMALE  I  25. 

G«AOE  21 

OTHERS  I  22. 

■tACK     I  60. 

$»'AmSM|  30. 

HALE       I  55. 

FEMALE  I  5?. 

--~-L  ' 
6«A0E  31 

OTHERS  I  13. 

I  26. 

S«>AN1SH|  2». 

I  33. 
FEMALE  I  35. 


■  I 


41 

OrHEffS  I  I. 

BLACK     I  20. 

SPANISHI  12. 

I  21. 

«HALC  I  12. 

GRAOC     51  ^ 

SLACK     I  8, 

SPAMISMI  6, 

cf"       ^  ^• 

FMALC  I  7. 

^^^'AL     I  14. 


U681 
1.660 
1*609 
1.649 
U680 
1.660 


1*782 

i«m 

1.747 
1.749 
1.729 
1.796 


1.695 
1.698 
1.634 
1.699 
1«693 
1.696 


1.990 
l.SOl 

i.rsi 

1.749 
1.899 
1.780 


1.862 
1.817 
1.857 
1.829 
1.849 


0.1272  I 
0.1454  I 
0.1221  I 
0.1945  I 
0.1291  I 
0.1921  I 


2.515 
2.407 
2.979 
2.492 
2.480 
2.455 


0.4400 
0.9595 
0.9289 
0.9859 
0.412) 
0.9954 


I 


LO 


1.4 

2.0 
2.0 
1.4 
2.0 
1.4 


"I  .  I 
I 

3.5  I 
3.2  I 
2*9  I 
3.2  I 
3.9  I 
3.5  I 


7 INC  ON  COMPUTER 
HCAN         ST-OEV  | 


RATE 
H6AN  ST-OEV 


1035.9 
1006.5 
1102.4 
1096.9 
1046.4 
1041.1 


141.400  I 
90.405  I 
46.854  I 

141.101  I 
■5.999  I 

117.976  I 


0.1968  I 
0.1529  I 
0.1456  I 
0.1514  I 
0.1464  I 
0.1488  I 


0.0859  I 
0.1444  I 
0.1289  I 
0.1298  I 
0.1255  I 
0.1246  I 


2.968 
2.169 
2.117 
2.215 
2.168 
2.191 


2.618 
2.779 
2.641 
2.  788 
2.599 
2.687 


0.5958 
0.2497 
0.2866 
0.9099 
0.9766 
0.9421 


0.5057 
0.5111 
0.5099 
0.5128 
0.4839 
0.5041 


1.1 
1.3 
1.5 
1.1 
1.3 
1. 


2. 
1. 
2. 
2. 
1. 
1. 


I 

3.3  I 
2.8  I 
2.7  I 
3.0  I 
3*3  I 
3.3  I 


0.078 
0.075 
0.069 
0.074 
0.077 
0.075 


806.8 
572.8 
714.1 
691.5 
622.9 
656.6 


I 

240.738  I 
210.406  I 
203.442  I 
223.719  I 
239.470  I 
233.381  I 


I 


3.5 
3.6 
3.9  I 
3.9  I 
3.6  I 
3.9  I 


1299.5 
1224.6 
1063.0 
1159.9 
1179.5 
1170.0 


391.565  I 
477.049  I 
253^.919  I 
364.342  I 
409.732  I 
389.605  f 


I 


MAGE  61 
SPANISMI 
I^EHALC  I 
TOTAL  I 


0.0 
0.1144  i 
0.1583  I 
0.1273  I 
0.1371  I 
0.1352  I 

I 

0.0518  I 
0.0  753  I 
0.0535  I 
0.0756  I 
0.0646  I 


3.600 
2.556 
2.988 
2.591 
3.014 
2.745 


2.825 
2.800 
2.657 
2.971 
2.814 


0.0 
0.5747 
0.8311 
0.6383 
0.7694 
0.7077 


0.5523 
0.5762 
0.4577 
0.6047 
0. 5405 


3.6 
1.7 
1.6 
1.6 
2.2 
1.6 


3.6  I 

3.7  I 
5.0  I 
3.7  I 
5.0  I 
5.0  I 


981.0 
699.3 
797.8 
765.6 
70  5.3 
743.  7 


1299*2 
1786.3 
1325.9 


I 


0.0 
199.635  i 
196.908  I 
183.626  I 
235.980  f 
202.688  I 


1. 
1. 
1. 


1.900 
1.900 
1.900 


0.0 
0.0 
0.0 


I 
I 
I 
I 


9.  100 
5.  100 
5.100 


504.592  I 
217.364  I 
535.854  I 
327.784  I 
466.736  I 


0.080 
0.091 
0.056 
0.078 
0.081 
0.079 


0.078 
0.101 
0.100 
0.  106 
0.087 
0.096 


0.170 
0.114 
0.152 
0.104 
0U173 
0.129 


0.000 
0.056 
0.073 
0.067 
0.070 


I 
I 
I 
I 


0.322 
0.322 
0.322 


0.042 
0.037 
0.026 
0.039 
0.037 
0.038 


0.074 
0.055 
0.061 
0.059 
0.065 
0.062 


0.040 
0.055 
0.055 
0.054 
0.050 
0.052 


0.0 

0.079 

0.118 

0.075 

0.112 

0.094 


0.039 
0.037 
0.048 
0.029 
0.038 


0.0 
0.0 
0.0 


i         Table  3-4  presents  the  rate  of  progress  by  grade  and  entry  level 

jfor  the  three  CAI  curriculums:    mathematics,  reading,  and  language,  i 

1  •  I 

Differences  among  the  curriculums  are  Immediately  apparent*    Rate  1 

1                                    •        -  i 

I                                                               '  ! 

<bf  progress  Is  lowest  In  reading  and  highest  In  language.    In  the  reading 

curriculum  only  the  first  two  entries  In  grade  4  and  the  last  three  In 

\  •         ■  1 

girade  6  are  above  •lOO,     In  the  language  curriculum  none  of  the  entries 
are  below  .100  and  13  of  the  27  entries  are  above  .200.     Rates  In  the 
mithematlcs  curriculum  are  lntermedlate-'"between  the  two  extremes, 
pirhaps  this  Is  what  the  conventional  wisdom  of  CAI  coordinators  was 
eJpresslng.     It  was  easy  to  get  through  the  language  curriculum,  harder 

i 

to|  get  through  the  mathematics  curriculum,  and  hardest  (or  slowest)  to| 
gel  through  the  reading  curriculum.     From  the  data  one  Is  unable  to  ! 
deiermlne  whether  It  Is  the  difficulty  of  Items,  the  underlying  software 
or  some  other  reason  that  makes  progress  In  the  reading  curriculum  so  i 

1  ■  i 

slow.  '  'i 

1  ■  ■  I 

T  ■  " " ^ '  i' 
1  .  1 

Summary  i 

i  ■  ■  i 

I  In  this,  chapter  we  have  seen  graphs  of  students*  progress  In  each  df 

the  three  CAI  curriculums.    We  have  seen  that  for  a  given  range  of  entry! 

level  scores~1.5  to  1.9  In  the  mathematics  CAI  curriculum — there  were  1 

Students  from  all  six  grade  levels  proceeding  through  the  curriculum  at 

varying  rates.    We  have  observed  that  the  rate  of  progress  Is  highest  in 

the  language  curriculum,  intermediate  in  the  mathematics  curriculum, 

and  slowest  in  the  reading  curriculum.    The  data  agree  with  observations 

from  coordinators  in  the  CAI  labs. 

i 

'  •        i  . 


Table  3-4 


Rate  of  Progress  by  Grade  and  Entry  Level  for  Three  CAI 
Cur.riculums ;     MATHEMATICS,  READING  AND  LANGUAGE. 


MATHEMATICS 

1.0- 

1.5- 

2.0- 

2.5- 

3.0- 

3.5- 

4.0- 

4.5- 

5.0- 

5.5- 

6.0- 

GRADE 

1.4 

1.9 

2.4 

2.9 

3.4 

3.9 

4.4 

4.9 

5.4 

5.9 

6.4 

1 

.086 

.075 

.076 

2 

.101 

.079 

.087 

.122 

.169 

.154 

3 

.095 

.096 

.099 

.116 

.130 

.161 

.230 

4 

.130 

.129 

.114 

.123 

.143 

.142 

.097 

5 

.070 

.079 

.092 

.106 

.109 

.126 

.127 

.108 

.125 

.113 

6 

.322 

.126 

.141 

.130 

.115 

.164 

.188 

.173 

.116 

.107 

6.5-  7.0-  7.5- 
6.9     l7.4  7.9 


.079 


READING 
GRADE 

1  .  ■    

2 
3 

4  .130    .102    .068    .073    .065    .068    .072    .016  .051 

5  .061     .083    .086    .064    .075    .081     .062    .067  .086 

6  .077     .089    .065    .065    .057    .064    .246    .122  .112 


LANGUAGE 


GRADE 

1 
2 


ERIC 


3 

.156 

.188 

.198 

.176 

.320 

4 

.212 

.264 

.236 

.296 

.175 

.137 

-5 

i 

.234 

.223 

.170 

.254 

.186 

.161 

.267 

.230 

6 

.226 

.163 

.184 

.161 

.164 

.166 

.280 

.259 

Chapter  .V 
CONCLUSIONS 

Part  3  of  the  final  report  has  described  briefly  the  three  curriculums 
used  in  our  longitudinal  study  of  CAI.     The  curriculums  were  found  to 
adapt  readily  to  the  entry  level  abilities  of  most  students.    The  mathe- 
matics strands Icurriculum  had  the  broadest  capability.     Students  in  grade 
4,  for  example,  were  placed  across  all  grade  levels  by  the  initial  rapid 
motion  phase  of  the  curriculum,  and  regular  progress  in  the  curriculum  was 
fast  enough  to  keep  students  motivated. 

Reading  and  language  CAI  curriculums  were  less  adaptable  to  students' 
entry 'level  abilities.    Some  fourth  grade  students  topped  out  of  the 
language  curriculum,  while  others  had  dif^i^tiity  moving  beyond  the  lowest 
level  of  the  reading  curriculum.     Progress  in  the  language  curriculum  was 
fast  enough  to  keep  students  motivated,  but  progress  in  reading  was  much 
slower. 

We  also  took  a  careful  look  at  students'  time  on  CAI  in  order  to 
determine  that  the  research  design  requirements  were  met.    Although  there 
was  considerable  variation  in  the  amount  of  time  on  CAI  for  individuals, 
times  for  different  CAI  treatment  groups  were  remarkably  alike.    We  also 
found  that  students  assigned  to  varying  levels  of  CAI  did,  in  fact, 
receive  varying  levels  of  treatment.    We  turn  now  to  Part  4  of  the  final 
report  and  an  examination  of  'the  effectiveness  of  the  CAI  curriculums. 
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PART  ^* 


COMPIHER-ASSISTED  INSTRUCTION  AND  COMPENSATORY  EDUCATION: 
'  THE  ETS/LAUSD  STUDY 


THE  EFFECTIVENESS  OF  CAI 


*This  appeared  as  Part  ^  of  M.  Ragosta^  P.  W.  Holland^  and  D.  T.  Jamison 
Computer-Assisted  Instruction  'And  Compensatory  Education:  The  ETS/LAUSD  Study. 
Final  Report  to  the  U.  S.  National  Institute  of  Education^  April  1982. 
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OViERVIEW 


in  part  4  of  the  Final  Report  we  turn  to  studies  of  the  effectiveness 
of  the  CCC  curriculums  as  used  in  CAI  labs  in  four  elementary  schools  in 
the  Los  Angeles  Unified  School  District.    Part  4  of  the  Final  Report  is 
organized  in  the  following  manner: 

In  Chapter  I  the  research  design  is  presented. 

.     In  Chapter  II  a  discussion  of  the  research  methodology  will 

present  the  underlying  statistical  model.     The  explication  of  a 
specific  analysis  is  used  to  illustrate  the  basic  procedures'.    ^  - 
Finally,  there  is  a  short  section  on  how  to  read  the  tables-nAsed 
to  report  the  treatment  effects.' 

.     In  Chapter  III  12  one-year  studies  of  CAI  are  summarized  by  grade 
level.    Only  within-CAI  analyses — the  randomized  parts  of  the 
research  design— are  reported.  . 

.     In  Chapter  IV  each  of  the  three  longitudinal  studies  is  summarized. 
Within-CAI  analyses  are  presented  first  but  analyses  based  on  cohort 
controls  and  comparison-school  students  are  included. 

.     In  Chapter  V  the  big  picture  is  presented.     Treatment  effects  in 
mathematics,  reading  and  language  arts  are  estimated  in  terms  of 
standard  deviations.     These  estimates  are  made  first  for  the 
one-year  within-CAI  studies,  then  for  the  longitudinal  within-CAI 
studies,  and  finally  for  the  cohort-control  and  comparison-school 
studies. 

In  Chapter  VI  we  summarize  our  findings  and  present  our  conclusions. 
Before  we  turn  to  the  effectiveness  data,  however,  let  us  consider 
what  it  is  that  we  are  studying.    We  are  estimating  the  effectiveness  of 
several  CAI  curriculums  produced  by  Computer  Curriculum  Corporation  and 
used  in  CAI  labs  as  a  pull-out  program  for  drill-and-practice  in  mathe- 
matics, reading,  and  language  arts.     There  are  many  versions  of  CAI;  the 
one  we  are  studying  represented  the  broadest  programmatic  approach  at  the 
time  of  this  project's  funding. 
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Chapter  I 

i  THE  RESEARCH  DESIGN 

The  research  design  determined  which  students  would — or  would 
not— receive  computer-assisted  instruction  during  each  of  the  four  years 
of  the  study.     It  further  determined  which  of  the  CAI  curriculums,  or 
Combination  of  curriculums,  a  student  would  receive.     It  also  provided 
for  three  kinds  of  control  groups. 

Six  schools  in  Los  Angeles  participated  in  the  study:    four  as 
schools  with  CAI  labfe  and  two  as  comparison  schools.    Students  in  the  two 
comparison  schools  provided  one  kind  of  control  group.    The  advantage  of 
the  comparison  group  was  that  data  could  be  collected  simultaneously  for 
CAI  students  at  a  particular  grade  level  and  students  at  the  same  grade 
level  in 'comparison  schools.    At  the  end  of  any  year  of  data  collection, 
CAl  students  could  be  directly  compared  to  comparison-school  students. 
There  were  obvious  disadvantages  to  using  comparison  schools:  differences 
in  populations  of  students,  leadership  styles  of  principals,  teachers  and 
teaching  practices,  and  the  overall  school  environment.     Better  control 
gi^oups  were  provided  from  within  the  CAI  schools. 

Approximately  half  of  the  students  in  the  four  CAI  schools  received 
computer-assisted  instruction  whilfe  half  did  not. 
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Grades 


1 


2 


3 


4 


5 


6 


Year 


one-year 
study 


Students  in  grades  2,  4  and  6  were  assigned  CAI  In  year  1;  their  cohorts 
In  grades  1,  3  and  5  were  not.    In  year  2,  first  graders  were  assigned 
CAI  along  with  third  and  fifth  graders  who  continued  to  receive  It.  In 
year  3,  CAI  was  limited  to  students  In  grades  2,  4  and  6.    In  year  4,  CAI 
was  continued  for  students  In  grades  3  and  5  and  a  special  one-year  study 
of  CAI  was  conducted  In  grade  4.    Students  In  grades  where  CAI  was  not 
assigned  became  the  second  kind  of  control  group— "the  cohort  controls. 
One  advantage  of  the  cohort  control  group  was  that  the  population  of 
students  and  the  school  environment  (principals,  teachers,  school 
practices,  etc.)  were  similar  to  those  of  the  CAI  students.     Ttie  disadvan- 
tage was  that  data  at  any  grade  level  was  collected  a  year  earlier  or 
later  than  data  for  CAI  students.    At  the  end  of  year  1,  for  example,  no 
cohort-control  data  was  available  for  grades  2,  4  and  6  where  students 
were  receiving  CAI.    Only  after  the  second  year  of  data  collection  was 
completed  could  CAI  students  be  compared  with  their  cohort  controls* 


The  comparison  of  CAI  students  with  their  cohort  controls  is  a  test  of 
CAI  used  as  a  substitute  for  part  of  a  teacher's  classroom  time. 

The  best  control  groups  were  provided  within  the  ranks  of  students 
receiving  computer-assisted  instruction.    These  control  groups  were 
achieved  by  random  assignment  of  students  to  their  CAI  curriculums, 
thus  creating  the  conditions  for  a  genuine  social  experiment.  Twelve 
one-year  studies  of  CAI  were  designed  to  evaluate  the  effectiveness  of 
the  CAI  curriculums  over  one  year  or  longer  periods  of  time.    The  twelve 
one-year  studies  are  outlined  in  Figure  4-1. 

i  ■        -  .  . 

!        In  year  1,  sixth-grade  students  were  randomly  assigned  within 
cllassrooms  to  receive  either:     two  session  of  mathematics  (MM),  two 
sessions  of  reading  (RR),  two  sessions  of  language  '(LL) ,  or  one  session 
of  reading  and  one  of  language  (RL).     The  MM  students  served  as  a  control 
group  for  those  CAI  students  receiving  reading  and/or  language  and, 
conversely,  the  RR,  LL  and  RL  groups  served  as  controls  for  students 
receiving  mathematics  CAI. 

Similarly,  students  in  grade  4  in  year  1  were  randomly  assigned  to 
receive  either:     two  sessions  of  mathematics  (MM) ,  one  session  of  reading 
and  ©ne  of  language  (RL),  or  one  session  of  mathematics  with  one  session 
where  reading  and  language  alternate  (MRL).     Students'  assignments  in 
grade  4  were  continued  into  grades  5  and  6,  and  new  students  were  randomly 
'  assigned  as. they  entered  the  system.     The  MM  group  served  as  a  control 
for  two  levels  of  CAI  reading  and  language  assignments  (RL  and  MRL), 
and  the  RL  group  served  as  a  control  for  two  levels  of  mathematics  CAI 
(MM  and  MRL). 


(NOTE:    M  -  M-theriatlcs  CM.  R  -  Re.dlng  CAI.  L  -  Language  CAI.  C  -  Reading  for  Comprehension  CAI) 


* 


Figure  4-1     CAI  TrMtment  Over  4  Tears 
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Students  In  grade  2  in  year  1  were  randomly  assigned  within  class- 
rooms to  either  one  or  two  sessions  of  mathematics  CAI.    This  was  the 
only  design  which  created  opposition  on  the  part  of  teachers  and  students 
and  it  was  never  repeated.    When  the  students  became  third-graders, 
they  were  reassigned  to  receive  either  two  sessions  of  mathematics  CAI 
daily  (MM)  or  one  session  of  mathematics  and  one  of  language  (ML).  The 
simpler  vocabulary  and  faster  movement  of  the  language  CAI  curriculum 
were  the  reasons  for  its  selection  over  the  reading  curriculum  for  this 
grade  level.     In  subsequent  years,  the  ML  students  received  reading  and 
language  CAI  (the  RL  group)  while  the  MM  group  continued  to  receive  two 
sessions  of  mathematics.     In  the  three-year  longitudinal  study — and  the 
two  two^6ar  studies — which  resulted  from  the  use  of  CAI  in  grades  3-5, 
the  MM  group  served  as  controls  for  the  ML-RL-RL  group  and  the  latter 
served  as  controls  for  the  MM  group. 

For  the  first-grade  students  who  became  part  of  the  CAI  study  in 
year  2,  random  assignments  within  classrooms  could  not  be  made  to  different 
curriculums  because  only  mathematics  CAI  was  available  in  grades  1  and  2. 
Random  assignments  to  difi^frent  levels  of  mathematics  CAI  were  not  made 
within  classrooms  because  of  the  disruption  caused  by  that  design  in  year 
1.     First-grade  classrooms  were  first  paired  on  the  basis  of  student- 
ability  and/or  other  variables,  and  then  one  member  of  the  pair  was 
randomly  assigned  to  CAl  while  the  other  served  as  a  control.    The  same 
procedure  was  used  subsequently  in  grade  2.     By  the  end  of  grade  2  there 

V 


were  4  kinds  of  CAI  conditions  among  the  students:     some  had  received  two 
years  of  mathematics  CAI,  some  had  received  CAI  only  in  grade  1»  some  had 
received  CAI  only  in  grade  2,  and  some  had  received  no  CAI  at  all.  In 
grade  3  in  the  final  year  of  the  study,  all  students  were  randomly 
assigned  within  classrooms  to  receive  either  one  session  of  mathematics 
or  one  session  of  language.     In  the  three-yeat  longitudinal  study  in- 
cluding students  in  grades  l-3»  controls  were  provided  by  levels  of  use 
of  the  mathematics  CAI  curriculum:     for  zero,  one,  two,  or  3  years* 

In  the  special  one-year  study  at  fourth  grade  in  the  final  year, 
students  were  randomly  assigned  to  one  10-minute  CAI  session  daily  of 
either  mathematics  (M),  reading  (RX,  language  (L)  or  a  newer  curriculum 
(available  at  one  school  only)  reading-f or-comprehension  (C).     In  this 
study  each  group  served  as  a  control  for  the  others. 

As  part  of  the  research  design,  all  students  were  pretested  and 
^posttested  each  year  of  the  study  with  standardized  achievement  tests; 
generally,  Iowa  Tests  of  Basic  Skills  in  the  fall  (September/October)  and 
the  Comprehensive  Tests  of  Basic  Skills  in  the  spring  (April/May). 


PRETEST 


POSTTEST 


GRADE  1 


CTBS  Form  S  Level  A 


CTBS  Form  S  Level  B 


2 


ITBS  Form  5  Level  7 


CTBS  Form  S  Level  C 


3 


ITBS  Form  5  Level  8 


CTBS  Form  S  lievel  I 


4 


ITBS  Form  5  Level  9 


CTBS  Form  S  Level  1 


5 


ITBS  Form  5  Level  10 


CTBS  Form  S  Level  2 


6 


ITBS  Form  5  Level  11 


CTBS  Form  S  Level  2 
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Except  for  the  fall  of  year  1,  all  students  were  also  pretested  and 
posttested  each  year  with  curriculum-specific  tests  (CSTs)  developed  for 
the  study  from  the  CAI  curriculums.     Both  the  standardized  tests  and 
CSTs  are  relevant  and  reasonable  instruments  for  measuring  treatment 
effects  but  they  may  be  viewed  as  providing  two'  kinds  of  information: 
the  former  give  a  general  treatment  effect  which  may  be  compared  with 
treatment  effects  in  other  studies  using  standardized  tests;  the  CSTs 
give  what  may  be  considered  the  upper  boundary  of  the  treatment  effect. 
Whereas  the  treatment  effects  arising  from  the  standarized  tests  might  be 
depressed *due  to  a  mismatch  between  the  CAI  curriculum  and  the  test,  the 
treatment  effects  using  the  CSTs  might  be  over inflated  because  control 
groups  were  not  exposed  to  the  CAI  material  in  the  classroom. 


'  Chapter  II 

RESEARCH  METHODOLOGY 

Our  basic  suimnary  measures  of  'the  effectiveness  of  the  various  CAI 

.  **  * 

curriculums  studied  in  this  investigation  are  estimated  treatment  effects 
that  have  been  adjusted  for  pretest  scores  knd  a  variety  of  other 
variables.     In  this  section  we  shall  give  a  careful  description  of  the 
regression  methodology  that  was  used,  to  obtain  these  estimates  throughout 
the  many  subanalyses  that  make  up  this  report.    We  shall  concentrate  here 
on  a  specific  analysis  that  we  will  call  the  Basic  Analysis.     The  Basic 
Analysis  formed  the  prototype  for  all  of  the  analyses  of  achievement  data 
in  this  report.     Specific  subanalyses.  required  certain  modifications  of 
the  Basic  Analysis,  and  these  modifications  are  described  in  the  sections 
that  give  the  subanalysis  results.  | 

The  Underlying  Statistical  Mo'del 

i  '^^ 

In  all  of  the  analyses  reported  here,  we  kre  interested  in  the 
relative  effects  of  exposure  (and  non-expos ure)j  to  various  CAI  curriculums 
on  student  outcomes  as  measured  by  certain  tests.    These  test  scores  are 
the  dependent  variables  in  our  analysis.    We  shall  denote  a  generic 
dependent  variable  by .  the  symbol  Y,  as  is  customary.    Howfever,  we  will  de- 
part somewhat  from  the  usual  regression  notation  because  we  wish  to 
emphasize  that,  in  an  experiment  like  the  present  one,  there  are  several 
versions  of  the  dependent  variable.     There  is  a  version  for  each  potential 
exposure  condition.     For  exarajple,  in  grade  4  year  1  there  were  three 
possible  combinations  of  CAI  curriculums  to  which  a  student  could  be 
assigned.     These  were  denoted  MM,  MRL  or  RL,  respectively.    Hence  for  each 
student  in  the  experiment  and  for  each  Y  there  are  potentially  three 
different  versions  of  Y  that  we  could  measure  —  denoted  Y^,  or  Y^. 
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The  value  of         is  the  value  of  Y  that  would  be  observed  if  the  student 
MM 

were  exposed  to  MM.     The  value  of  Y^^^^  is  the  value  of  Y  that  would  be 
observed  if  the  same  student  were  exposed  to  MRL.     Similarly  for  Y^^. 
If  the  CAl  curriculums  differ  in  their  relative  effectiveness,  then  for 
each  student  the  values  Y^^,  Y^^^^  and  Y^^  would  not  be  expected  to  be  the 
same.     For  example,  if  Y  denotes  the  score  on  a  mathematics  test,  then  we 

would  expect  that  Y      >  Y        >  Y      for  each  student  in  the  experiment, 

MM         MicvL  KJL 

since  this  ordering  is  the  same  as  the  order  of  the  amount  of  exposure  to 
the  CAI  Math  curriculum.    However,  it  is  impossible  to  observe  more  than 
one  of  these  three  versions  of  Y  for  each  student  in  the  type  of  experiment 
we  are  discussing.    Hence,   the  only  information  that  we  have  available  for 
a  given  subject  ^nd  a  given  dependent  variable  Y  is 

(Yg,  S,  X),  (1) 
where  S  denotes  the  CAI  (or  non-CAI)  condition  to  which  the  student  was 
exposed,  Y^  denotes  the  corresponding  Version  of  the  dependent  variable  and 
X  denotes  a  vector  of  covariates  which  are  either  measured  prior  to  the 
exposure  to  the  CAI  curriculum  (e.g.,  pretest  scores)  or  not  affected  by 
it  (e.g.,  gender  or  ethnicity).     Since  X  is  not  affected  by  S,  it  does  not 
have  several  versions  in  the  way  that  Y  does. 

We  can  not  directly  compare  the  values  of  Y^  and  Y^^^^  for  each 
student,  but  we  can  estimate  the  average  value  of  Y      and  of  Y      in  a 

I  MM  KL 

given  population  of  students  and  compare  these  averages.     Our  ability  to 
estimate  these  averages  depends  on  two  things  —  random  assignment  of 
students  to  curriculums  and  the  use  of  covariates.    We  had  one  or  both  i 
of  these  factors  available  to  us  thrbughout  the  entire  study. 


ERIC 
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Let  us  denote  the  operation  of  taking  a  population  average  of.  d 
variable  Y  by  the  notation  E(Y),  (or  by  Ep(Y)  If  It  Is  necessary  to  be 
expll^clt  about  thie  population,  P).     It  is  also  necessary  for  .us  to  have 
a  notation  for  conditional  population  averages,  that  Is  averages  of  the 
values  of  one  variable  among  only  those  students  with  a  given  value  of 
another:  varlab^le.     We  use  the  standard  notation  E(y|X)  to  denote  the 
conditional  average  of  Y  for  each  fixed  value  of  X.    We  will  make  use 
of  the  following  fundamental  equation  that  relates  population  averages 
and  conditional  population  averages: 

E(Y)  «  E{E(y|X)}.  (2) 
The  content  of  formula  (2)  is  very  simple  —  the  population  average  of 
Y  can  be  computed  by  first  averaging  Y  for  each  fixed  value  of  X  and 
then  averaging  these  cdhditional  averages  using  the  proper  weights  that 
come  from  the  distribution  of  X  in  the  population. 

The  i>opulation  treatment  effect  (T.E.)  of  an  exposure  condition, 

say  MM,  relative  to  a  second  exposure  condition,  say  RL,  is  the  average 

of  the  Y^^,  -  Y„^  differences  over  the  population,  i.e. 
JMM  RL 

T.E.  =  E(Y^-  Y^).  (3) 
However,  as  we  said  earlier ,  since  we  can  not  compute  Y^  -  Y^^  for  any 

single  student,  the  treatment  effect  in  (3)  must  be  estimated  via 
indirect  means.     The  first  step  along  this  path  is  to  use  the  fact  that 
the  ''average  of  a  sum  is  the  sum  of  the  averages"  to  obtain  the  equation 

X.E.  ^  E(Y^)  -  E(Y^)  =  E(Y^  -  Y^) .  '  '  (4) 

Therefore  the  problem  of  estimating  the  treatment  effect  in  (3)  has  been 
reduced  to  estimating  the  two  means  ECY^)  and  E(Yj^)  over  the  population 
of  students. 

:    Before  going  on  to  show  how  one  estimates  E(Y^)  and  E(Yj^)  we 

223 
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briefly  discuss  our  choices  of  treatment  and  control  comparisons. 

We  often  have  more  than  two  CAI  exposure  conditions  in  the  substudies 

of  this  report*    When  this  happens  there  is  no  unique  way  to  define  the 

treatment  effects*    We  have  adopted  the  convention  of  taking  one  of  the 

exposure  conditions  as  a  control  and  computing  all  of  the  treatment  effects 

relative  to  it*     For  the  grade  4  year  1  example  this  means  that  when 

the  dependent  variable  is  a  math  test  the  treatment  effects  to  be 

estimated  are 

and 

} 

These  effects  are  meaningful  in  the  context  of  this  study  because  the 
amount  of  exposure  to  the  CAI  math  curriculum  is  0  for  RL,  10  minutes 
per  day  for  MRL  and  20  minutes  per  day  for  MM.  Hence  we  would  expect 
the  effect  in  (5)  to  .exceed  the  one  in  (6). 

For  reading  and  language  dependent  variables,  a  different  set  of 
treatment  effects  are  relevant.    These,  are 


and 


Our  general  rule  for  choosing  treatment  effect^  is  to  use  as  the 
control  the  exposure  condition  with  the  -lowes t  amount  of  exposure  to  the 
curriculum  tested  by  the  dependent  variable.     Thus,  all  of  the  treatment 
effects  in  this  study  are  interpretable  as  the  average  nuxnber  of  test 
items  correctly  answered  due  to  the  increase  in  exposure  to  a  specific 
CAI  curriculum.    We  now  return  to  our  discussion  of  the  estimation  of 
the  treatment  effects. 
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It  is  one  thing  to  set  as  a  goal  the  estimation  of  the  treatment 
effects  given  in  (5)-(8),  but  quite  another  to  design  a  study  capable 
of ^ doing  that.     Critical  to  our  ability  to  estimate  these  treatment 
effects  are  two  key  features  of  the  present  study  —  randomization  and 
the  availability  of  coyariates.    We  discuss  the  role  of  these  features 
now. 

When  we  can  only  observe  (Yg,  S,  X)  for  each  student  as  defined  in 

(2),  where  X  -is  the  vector  of  covariates,  S  the  exposure  condition  to  ' 

which  the  student  is  exposed  and  Yg  the  value  of  either  Y^,  ^MEIL 

Y^,   (depending  on  the  value  of  S)  ,  it  is  not  directly  possible  to  estimate 

E(Y.-,v)  and  E(Y-,-).    For  exainple,  suppose  we  computed  the  average 
MTl  MKL  KL 

value  of  Y^-.,  for  all  those  students  exposed  to  MM  (i.e.,  for  whom  S  =  MM). 
MM 


This  is  an  estimate  of 


E(Yj^|S  =  MM)  •  (9) 


and  not  an  estimate  of  E(Y^j^).     The  relationship  between  E(Y^)  and 

E(Y.-..|S  «  MM)  can  be  expressed  as 
rlPl' 


E(Ymm)  -  E(Y^|S  -  MM)  P(S  «  MM) 


MM^      ^  MM' 

^    +  ^^^mm'^  ^  ^ 

Hence  wis  see  that  in  order  to  use  the  data  that"  are  collected  to  estimate 
the  parameter  of  interest,  a  study  must  be  designed  to  allow  us  to  make 
reasonable  assumptions  about  certain  quantities  or  parameters  that  are 
inherently  inestimable.     In  (10)  the  inestimable  parameter  is 

E(Y^|S  7^  MM)  (11) 
which  is  the  average  value  of  Y.^.  for  those'  students  in  the  population 

MM 

exposed  to  some  CAI  curriculum  other  than  MM.     Since  we  can  only  observe 
for  students  exposed  to  MM,  the  quantity  in  (11)  can  not  be  estimated 

MM 

directly.     This  does  not  mean  that  tlxe  quantity  in  (11)  is  meaningless. 
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For  example,  in  a  randomized  study,  the  assignment  of  students  to 

curriculuma  is  designed  to  be  statistically  independent  of  any  dependent 
variable.    A  mathematical  consequence  of  this  statistical  Independence  is 
that 

.E(Y^|S      MM)  =  E(Y^|S  «  MM).  (12) 

Equation  (12)  says^  that  the  average  value  of         is  the  same  for  those 
students  assigned  to  MM  and  those  assigned  to  the  other  curriculums.  If 
equation  (12)  is  applied  to  (10>  we  obtain  the  following  basic  fact 
about  a  randomized  experiment: 

"  ^^^MM^^S  ■  '  ^^^^ 

The  left  side  of  equation  (13)  is  the  parameter  we  wish  to  estimate 

while  the  right  side  of  (13)  is  the  parameter  we  can  estimate.  Randomiza- 
tion makes  them  equal.' 

A  second  device  that  helps  us  towards  our  goal  of  estimating 

EC^%«>t)i  etc.  is  the  use  of  the  covariates,  X,  which  appear  in  (1).  The 
MRL 

covariates  used  in  this  study  are  pretest  scores  and  certain  demographic 
characteristics  —  gender,  and  ethnicity.    The  regression  of  Y^^  on  X 
for  the  students  exposed  to  MM  is  an  estimate  of  the  conditional  population 
average  of  Y^.,  given  the  value  of  X  and  S  «  MM.    This  is  represented 

/  MM 

mathematically  by  * 

E(Y^[S  -  MM,  X).  (14) 

There  Is  ^an  equation  that  relates  the  regression  function  In  (14)  to  the 

quantity  that  we  are  interested  In  estimating,  I.e.     E(Y^).  This 

nrl 

equation  Is  like  the  equation  In  (^-0) .     It  Is 

E(Y^)  -  E{E(Yjj^|S  -  MM,'X)  P(S  -  MM|X)} 
+  E{E(Y^IS  ^  M,  X)  P(S  ^  MM)X)}. 


(15) 


MM' 


■  23^ 


Again,  we  see  that  the  relation  between  E(Yj^)  and  the  regression 

function  that  we  can  estimate  involves  an  inherervtly  inestimable  quantity. 

This  time  the  inestimable  quantity  is  '  . 

E(Y^|S  9^  MM,  X)  (16) 

which  is  the  regression  of  Y.^,  on  X'for  those  students  exposed  to  a 

MM 

curriculum  other  than  MM. 

2 

If  X  predicts  Y^^  well,'  (i.e.,  has  a  high  R    value)  or  if  S  is 
MM 

independent  of  Y^. because  of  randomization,  we  are  often  willing  to 
assume  that  the  regression  function  in  (16)  equals  the  one  in  (14). 
When  this  is  true  (15)  reduces  to 

E(Y^)  =  E{E(Y^|S  =  MM,  X)}.  (17)' 
Formula  (17)  does  not  assume  that  the  regression  function  of  Y^^  on  X 
is  linear,  but  for  simplicity  we  will.     If  (14)  is  a  linear  regression 


then  it  is  of  the  form 

E(Y^|S  =  MM,  X)  «  a  +  b  X  (18) 
MM' 

where  a  is  a  scalar  and  b  is  a  vector  compatible  with  X.    We  have 

E(Y^)  =  a  +  b  E(X>.  (19) 

Equation  (19)  gives  a  formula  for  E(Yj^)  in  terms  of  the  regression 
of  ^MM       ^  group  exposed  to  MM,  and  the  average  of  X  over  the 

population.     Correspondingly,  if  we  can  assume  that 

E(Yj^|S  9^  RL,  X)  «  E(Yj^|S  «  RL,  X)  =  a*  +  b*  X  (20) 

then  we  have 

E(Yj^)  =  a*  +  b*  E(X).  (21) 
The  treatment  effect  of  MM  relative  to  RL  is  then 

T.E.  -  E(Y^)  -  E(Yj^)  =  a-a*  +(b-b*)  E(X).  (22) 
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A  special  simplification  occurs  when  the  regressions  in  (19)  and  (21)  are 
"parallel,"  i.e.,  when  b  «  b*.  In  this  case  the  treatment  effect  reduces 
to 

T.E.  «  a-a*  (23) 
which  is  the  difference  in  intercepts  for  the  two  regressions. 

Equation, (23)  is  the  basis  for  the  way  in  which  we  performed  the 
analysis  in  this  study.    We  may  summarize  the  assumptions  that  lead  to  it 
as  follows: 

(Al)         Conditional  independence  of         and  S  given  X, " 

MM 

E(Y^lS  .  MM.  X)  =  E(Y^lS  ^  MM.  X). 

(Similarly  for  the  other  exposure  condj(^ions,  MRL  and  RL.) 

(A2)         Linear  regressions,  e.g. 

E(Y^|S  =  MM,  X)  =  a  +  b  X- 
MM' 

(A3)         Parallel  regressions,  e.g.     the.  regressions,  of  Y^^,  and 

Y      on  X  all  have  the  same  slopes,  but  may  have  different 

RL  ^ 

intercepts. 

Assumptions  (A2)  and  (A3)  can  be  and  were  checked  on  the  data  for  their 
adequacy.     Assumption  (Al)  can  be  insured  by^  the  use  of  randomization 
and/or  the  use  of  covariates  which  are  well  correlated  with  the  dependent 
variables. 

The  Basic  Analysis 
We  now  discuss  the  specific  analysis  used  in  grade  4,  year  1  of 
the  study.     This  is  the  Basic  Analysis  referred  to  earlier  and  forms 
the  prototype  for  all  of  the  other  analyses  performed  in  this  part  of 
the  report. 
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There  were  three  CAI  curriculums  available  in  grade  A  mathematics 
(M),  reading  (R)  and  language  (L) .     Students  were  assigned  randomly  to  one 
of  three  CAI  curriculum  combinations  in  grade  4,  year  1.    These  are  denoted 
MM,  MRL  and  RL: 

MM  »=  Two,  10-minute  sessions  of  mathematics  CAI  per  day, 
RL  «  One,  , 10-minute  session  each  of  reading  CAI  and 

language  CAI  per  day  (total  of  20  minutes  per  day). 
MRL  «  One,  «10-minute  session  of  mathematics  CAI  per  day 

and  one,  lO-minute  session  of  either  reading  CAI  or 
language  CAI  (alternating  weekly)  per  day  (total  of 
20  minutes  per  day). 
The  assignment  of  students  to  the  treatment  conditions  was  random  within 
classrooms.     Thus,  each  fclassroom  may  be  viewed  as  a  separate  population, 
P^,  in  the  analysis.    However,  the  sample  sizes  at  the  classroom  level 
are  quite  small,  and  it  is  customary  in  such  circumstances  to  "pool"  the 
analysis  across  the  classrooms.    By  "pooling"  we  do  not  mean  simply 
treating  all  the  classrooms  together  as  one  big  sample  in  which  we  ignore 
the  classroom  a  student  comes  from.     Instead  we  "pool  by  parallel  regres- 
sions".    This  is  done  as  follows.     In  the  'i—  classroom,  P^,   the  mean  of 

Y  ^,  is  denoted  E„  (Y^^J .     Equation  (19)  should  then  be  written  to  show 
MM  MM 

the  dependence  of  the  regression  coefficients  on  i,  i.e. 

"  "  \  ^  h  ^^^^ 

Similarly,  for  equation  (21) .    Under  assumptions  (Al)  ,   (A2)  and  (A3). 
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the  treatment  eff^^t  of  MM  relative  to  RL  in  classroom       is  a^  -  a^*. 
When  we  pool  by  parallel  regressions  we  assixme  that  the  treatment 
effects  in  each  classroom  are  equal,  i.e.  that 

for  each  i.    We  also  assxime  that  the  slo^s        in  all  the  classrooms  art 
equal.    A  corresponding  ^assumption  is  made  regarding  the  treatment 
effects  of  MEIL  relative  to  RL.     These  assumptions  in  conjunction  with 
(Al),  (A2),.and  (A3)  lead, to  these  three  parallel  regression  equations: 

Ep^(Yj^IS  =  RL,  X)  =  a^*  +  b  X 

These  regression  equations  are  estimated  all  at  once  by  a  single 

regression  analysis  which  fits  the  following  model: 

Y  «=  a  +  Z  c^Cl/+  dMALE  +  Z  e.ETH. 

i    i    i  ,   j    J  J 


(27) 


•I-  Z  b,  PRT,.  +  Xj^jjMM  +  t»„tMRL 


k 

In  (27)  CL^  is  an  indicator  (zero/one)  variable  for  students  in 

(CL    -  1  for  students  in  P.);  MALE  is  an  indicator  variable  for  gender 
i  ^ 


(MALE  =  1  for  males);  ETH^  is  an  indicator  variable  for  the  j  ethnic 


th 

itor  variable  tor  tne  j- 
category;  PRT,   is  the  value  of  the  k-^  pretest  score  used  in  the  analysis 
as  a  covariate;  MM  is  a  treatment  indicator  for  students  assigned  to  MM. 
Similarly  for  MRL. 
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The  identification  of  the  regression  parameters  in  the  three 


equations  in  (26)  with  the  estimated  coefficients  in  (27)  is  as 


follows  :^ 


a 


i 


=  a  +  c 


i 


for  all  but  the  last  classroom, 


4; 


b 


*  (d,  e. ,  e 


>  •  •  • 


) 


T^,  and  T^_.   in  (26)  correspond  exactly  to  the  quantities  with 
MM  MKL 

the  same  labels  in  (27). 

Table  4-1  gives  the  estimated  values  for  the  coefficients  in 
(27)  and  their  associated  t-values  for  the  dependent  variable, 
Y  «  Total  Score  on  fourth  grade  Math  CST.     From  the  values  in  Table 

4-1  we  can  form  the  three  estimated  regression  equations  from  (26) 
that  correspond  to  each  classroom.     For  example,  for  the  first  class- 
room (i.e.,  CLASS  107)  the  three  equations  are: 

20.52  +  14.51  -  1.36  MALE  -10.73  ETH^  -  3.45  ETH2  +  0.87  PRT^  +  1.56  PRT2 
20.52  +  5.25  -  1.36  MALE  -  10.73  ETH^  -  3.45  ETH2  +  0.87  PRT^  +  1.56  PRT2 
20.52  1.36  MALE  -  10.73  ETH^  -  3.45  ETH2  +  0.87  PRT^  +  1.56  PRT^. 

These  three  estimated  regression  equations  are  the  result  of  a 
pooled  regression  and,  as  such,  are  not  the  equations  that  would  be 
obtained  if  a  separate  analysis  were  made  of  the  three  treatment  groups 
in  the  first  classroom.    The  results  of  the  pooled  regression  are 
preferred  when  the  samples  in  each  treatment  group  in  each  class  are 
small,  because  they  are  more  accurate  than  the  results  of  separate 
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Table  4-1 


Example  showing  the  results.  o£  a  pooled,  regression  analysis 
to  estimate  treatment  effects  for  grade  4  year  1. 


Dependent  variable 


CST  Math  Total. 


Variable 
Name  in 
Formula  (27) 


Coefficient 
in 

Formula  (27) 


Variable  Name 
in  Regression- 
Output  


Estimated 
Coefficient 


t- value 


— 

a 

PSEUDO 

■32.11 

4.18 

^1 

CLASS  107 

-11.59 

-1.55 

^^2 

CLASSICS 

6.38 

0.87 

^^3 

CLASS 2 16 

-Id. 50 

-1.52 

CL, 
A 

CLASS218 

-4.30 

-0.55 

CL5 

CLAS  8^301 

-5.86 

-0.96 

^6 

CLASS 308 

-7.27 

-1.09 

CLASS322 

-10.79 

-1.80 

CLASS 329 

-17.65 

-2.92 

CLASS414 

-4.57 

-0.76 

MALE 

d 

MALE 

-1.36 

-0.53 

ETH^ 

^1 

BLACK 

-10. 73 

-1.99 

ETH2 

'  .^2 

SPANISH 

-3.45 

-0.71 

PRT^ 

\ 

4M1.  .RWl 

0.87 

2.65 

PRT2 

>2 

4M2.  .RWl 

1.56 

3.67 

MM 

MM 

14.51 

4.52 

MRL 

^MPT 

MRL 

5.25 

175, 


0.580,      Std.-err.  reg.  =16.12 
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analyses.    For  example,  the  first  classroom  contains  18  students  dis- 
tributed into  the  three  CAI  exposure  conditions,  whereas  the  regression 
results  in  Table  4-1  are  biased  on  175  students  from  10  classrooms. 

When  using  pooled  regression  analyses  as  we  do  throughout  this 
report,  it  is  necessary  to  make  certain  checks  on  the  fit  of  the 
estimated  equation  to  insure  that  the  benefits  of  the  pooling  are  not 
lost  because  of  an  increase  in  bias  in  the  estimates.    We  made  these 
checks  by  routinely  testing  for  interactions  between  the  treatment 
indicator  variables  and  the  other  independent  variables  in  the  regres- 
sion analysis.  .  In  almost  all  cases  we  found  no  significant  interactions 
that  would  lead  us  to  change  our  method  of  estimating  the  treatment 
effects.  . 

The  Format  of  the  Tables  used  to  Report 
Estimated  Treatment  Effects 

We  have  adopted  a  consistent  format  for  displaying  the  estimated 
treatment  effects  for  all  of  the  substudies  discussed  in  this  part  of 
the  repoijt^     In  order  to  help  the  reader  understand  the  relevant 
information  in  these  tables  we  give  here  a  brief  description  of  the 
example  in  Table  4-2. 

The  heading  indicates  what  type  of  substudy  is  described  by  the 
table  —  a  one-year  analysis,  a  longitudinal  analysis  or  a  comparison 
of  CAI  with  non-CAI  students,  either  cohort  controls  or  comparison- 
school  controls*.    The  heading  also  gives  the  grade  level  and  year  or 
years  of  the  study,  as  well  as  the  type  .of  test(s)  used  as  the  dependent 
variable  (CTBS.or  CST)  and  the  subject  area.    Each  table  refers  to 
a  single  subject  area  (mathematics,  reading  or  language)  as.  the 
dependent  variable. 
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The  body  of  the  table  is  divided  into  columns  for  (a)  the  name  of 


the  sub-tests,  (b)  the  treatment  group,  (c)  the  N's  for  the  treatment 
group,  (d)  the  estimated  treatment  effects,   (e)  their  associated 
t-statistics,   (f)  the  posttest  means  for  each  treatment  group,  (g)  the 
posttest  standard  deviation  in  each  treatment  group  and  (h)  the  residual 
standard  deviations  from  the  regression  analysis  used  to  estimate  the 
treatment  effects,  and  (1)  the  standardized  treatment  effect. 

The  rows  of  the  table  are  grouped  according  to  the  subtests  and 
the  total  for  the  given  tested  subject  area  described  by  the  table. 
Each  of  these  groups  of  rows  consists  of  a  single  row  for  each  experi- 
mental group  in  the  analysis. 

The  treatment  effect  for  the  "control"  in  each  analysis  is  zero 
by  definition  and  this  is  indicated  by  a  dash  (-)  in  Table  4-2  for 
RL  students.    The  treatment  effects  for  the  experimental  conditions 

can  be  interpreted  as  mean  numbers  of  correct  items.     The  MM  group 

■J 

on  Subtest  A  answered  9.29  more  questions  correctly — on  the  average — 

than  did  RL  students,  adjusting  for  covariates.    The  degrees  of 

freedom  for  the  t-statistic  are  the  same  for  each  subtest  of  the  CST 

or  CTBS  and  are  given  in  the  heading.    The  "residual  is  defined 

as  the  square  root  of  the  error  mean  square  from  the  regression  analysis. 

The  standardized  treatment  effect  in  the  last  column  is  the  treatment 

effect  divided  by  the  residual  standard  deviation  and  can  be  interpreted 

as  a  treatment  effect  in  terms  of  standard  deviations.  Interpretation 

of  the  standardized  treatment  effect  derives  from  norms  for  the  rate 

of  achievement  growth  per  month  of  typical  schooling.    At  most  elementary 

school  grades  the  difference  between  the  average  student  at  the  beginning 


ERIC 


23j 


and  end  of  the  same  grade  is  about  1.00  standard  deviations One  month 
in.. flchool. accounts,  on  the . average  for  a  growth  of  0.10  standard  ; 
deviation  units.    In  Table  4-2,    the  treatment  effect  for  the  MM  group 
on  Subtest  A  when  divided  by  the  residual  S.D.  indicates  that  MM  students 
adjusted  mean  scores  are  about  one  standard  deviation  higher  than>ad^ 
justed  mean  scores  for  RL  students.  (9.29/9.38  «  .99)    This  me«ic — 
the  standardized  treatment  effect — is  used  whenever  treatment  effects 
are  to  be  averaged  across  studies.  > 
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Table  4-2 

One-Year  Analysif^  for  GRADE  ■  4  YEAR  ■  !• 

Posttesc  -  GST  AND  GTBS  Subject  •  MATKEMATIGS. 


Treatment  effects 

and  posttcst  means 

by  treatment  croup 

Schools  ^ 

GST 

Treatment 

Treatment / 

(t) 

Posttest 

Residual 

Standardized 

/Group 

N 

Effect  / 

158df . 

_X  

SD 

SD 

T .  E . 

/ 

PART  A 

MM 

CO 

C\  Oft 

9. 29 

4. 98 

35.94 

14.65 

9.38 

0.99 

MRL 

66 

3.06 

1.73 

29.76 

10.33 

0.33 

RL 

57 

27.35 

11.79 

PART  B 

MM 

' 

.52 

5.23 

3.25 

23.92 

12.39 

8.08 

0.65 

MRL 

66 

2.19 

1 . 44" 

20.29 

10.33 

0.27 

RL 

57 

18.75 

9.80 

MATH 

MM 

52 

14.51 

4.52 

59.86 

25.44 

16.12 

0.99 

TOTAL 

MRL 

66 

5.25 

1.73 

50.04 

23.04 

0.33 

RL 

57 

46.10 

20.41 

GTBS 

Treatment 

Treatment 

(t) 

Posttest 

Residual 

Standardized 

Group 

N 

Effect 

180df 

X 

SD 

SD 

T.E. 

COMPUTATION 

MM 

63 

-0.11 

-0.07 

35.57 

11.18 

8.14 

-0.01 

'  MRL 

67 

-2.19 

-1.48 

33.16 

11.11 

-0.27 

RL 

■  68 

35.71 

10.57 

GONGEPTS 

MM 

63 

-1.24 

-1.81 

15.90 

6.47 

3.73 

-0.33 

MRL 

67 

-0.95 

-1.40 

15.94 

5.48 

-0.25 

RL 

63 

17.44 

5.53 

APPLIGATIONS 

MM 

63 

-1.00 

^1.25  ' 

13.33 

7.16 

4.36 

-0.23 

MRL 

67 

-1.35 

-1.71 

12.79 

6.60 

-0.31 

RL 

68 

15.09 

6.29 

MATH 

MM 

63 

-2.34 

-1.01 

64.60 

22.16 

12.69 

-0.18 

TOTAL 

MRL 

67 

-4.50 

-1.95 

61.90 

20.82 

-0.  35 

RL 

68 

68.24 

20.64 
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     -  The -Roles-  of  CSTa..and  CTBS   

The  roles  of  the  CSTs  and  the  CTBS,  when  used  as  dependent  variables 
in  our  analyses,  are  viewed  by  us  as  somewhat  different •    The  subtests 
in  both  sets  of  tests  are  relevant  and  reasonable  instruments  for  measuring 
student  performance  in  their  respective  areas.    However,  because  the  CSTs 
were  constructed  deliberately  from  the  items  in  the^  CCC  curriculums,  their 
main  role  in  our  analyses  is  to  measure  the  extent  to  which  each  of  the 
CAI  curriculums  altered  the  students'  performance  on  the  material  used  in 
each  CAI  curriculum.     In  other  words,  the  CSTs  are  6urricu^.um  specific 
and  are  aimed  at  measuring  whether  or  not  each  CAI  curriculum  was  successful 
in  improving  the  performance  of  the  students  on  the  material  in  which  it 
was  drilling  them. 

The  CTBS,  on  the  other  hand,  is  the  standardized  test  used  by  the 
LAUSD  for  measuring  student  progress  from  all  types  of  schooling 
experience  —  not  merely  CAI  experience.     It  is  a  measure  that  is  not 
directly  tied  to  the  CCC  curriculum.     Student  improvement  on  the  CTBS 
that  is  due  to  exposure  to  a  CAI  condition  in  our  experiment  can  be  used 
to  estimate  the  effect  of  these  CAI  curriculums  on  standardized  test 
scores  more  generally. 


Chapter  III 
ONE-YEAR  EXPERIMENTAL  STUDIES  • 

 In  ■th±&  section  of  the  report  we  examine  the  effectiveness  of  the 

CAI  curriculums.    For  each  of  the  one-year  studies  in  the  research 
design,  we  will  suimaarize  the  result  by  grade  level.     Furthermore,  in 
this  chapter  we  will  examine  only  the  within-CAI  experimental  comparisons. 
When  we  restricted  intervention  to  within-CAl  comparisons,  then  most 
students  in  the  several  samples  had  about  the  same  length  of  axposure  per 
year  to  at  least  one  CAI  curriculum.     In  Table  4-3  all  the  one-year 
studies  are  tabulated,  and  the  CAI  exposure  conditions  summarized.  Some 
one— year  studies  were  designed  to  measure  the  one-year  effect  of  CAI  and 
some  were  a  part  of  a  longitudinal  study  which  was  designed  to  assess  the ^ 
long-term  effect  of  CAI  exposure. 

All  the  students  in  the  study,  regardless  of  which  of  the  six 
schools  they  were  in  and  whether  or  not  they  were  exposed  to  one  ot 
more  CAI  curriculum,  took  a  pretest  of  either  the  ITBS  or  CTBS  (depending 
on  the  grade  level)  and,  after  the  first  year,  a  curriculum-specific  test 
(CST)  before  the  onset  of  the  CAI  each  fall.     They  were  also  administered 
both  the  CTBS  and  the  CST  posttests  in  the  spring.    Other^ information 
such  as  sex  and  ethnicity  for  each  student  was  also  obtained.  The 
information  collected  in  the  study  is  summarized  in  Part  2  of  this  report 
together  with  a  description  of  the  overall  design  of  the  study. 

The  basic  regression  analysis  used  to  estimate  treatment  effects  is  , 
similar  for  all  the  one-year  studies.     It  is  described  in  Part  4,  Chapter 
II.     The  purpose  of  the  regression  analysis  is  to  estimate  the  CAI  treatment 
effects  relative  to  a  particular  control  group  after  adjusting  for  all 


i 

4 


CAI  Treatment  Over  k  Years 


(NOTE:  M 
Grade  1 

-  Mathematics  CAI,  R  -  R 
Grade  2 

eadlng  CAI,  L  -  Language 

Grade  3 

 s  ■  

CAI,  C  -  Reading  for  Co 
Grade  4 

mprehenslon  CAI) 
Grade  5 

V 

Grade  6 

1 

ro 

ON 
1 

Year 
1 

1977 

Random  within  Class 

Random  within  Class 

Random  within  Clasa 

•  M:    7  mlnutea 
dally 

vs. 

MM:  14  minutes 
dally 

MM:  20  minutes  daily 
vs. 

RL:  20  minutes  daliy 
vs. 

MR/L:  20  minutes  dally 

MM:  20  mlnutea  da^ly 
vs. 

RR:  20  mlnutea  dally 
vs. 

LL:  20  minutes  dally 
vs. 

RL:  20  mlnutea  dally 

21- 

Year 
2 

1978 

Random  by  Class 

M:    7  Blnutea 
aaiiy 

vs. 

No  CAI 

Random  within  Class  ' 

ML:  2q  mlnutea  dally 
vs. 

MM:  20  minutes  dally 

Random  Within  Clasa 

MM:  20  minutes  dally 
va. 

RL:  20  minutes  dally 
vs. 

MR/L:  20  mlnutea  dally 

Year 

3 

1979 

Random  bv  Class 

M:  10  mlnutea 
dally 

vs. 

No  CAI 

Rjindom  within  Class 

RL:  20  minutes 
vs. 

MM:  20  minutes 

Random  bv  Clasa 
T.  &  Th.  vs.  M/W/F 



Random  within  Claaa 

MM:  20  mlnutea  dally 
va. 

RL:  20  mlnutea  dally 
vs. 

MR/L!  20  minutes  dally 

Year 

1980 

t 

Random  with  In  Clasa 

Random  within  Class 

Random  within  Class 

2 

M:  10  mlnutea  dally 
vs. 

L:  10  mlhutes  dally 

M:^10  minutes  dally 
va. 

R:  10  minutes  dally 
vs. 

L:  10  minutes  dally 
va. 

C:  10  minutes  dally 

RL:  20  minutes  dally 
vs. 

MM:  20  minutes  dally 
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the  pretests  and  demographic  differences.    The  choice  of  specific  control 
groups  depends  on  the  particular  dependent  variable  being  examined.  As 
we  indicated  in  part  4,  section  II,  our  general  rule  was  to  use  as  the 
control  the  CAI  treatment  with  the  least  exposure  to  the  curriculum 
tested  by  the  given  variable.  < 

For  each  of  the  six  grade  levels,  we  will  first  summarize  the 
studies  and  the  CAI  curriculums  compared  over  the  four  years.  Even 
though  the  original  research  plan  called  for  complete  randomization 
wherever  possible,  some  deviation  from  complete  randomization  did  occur 
for  numerous  reasons.     For  this  reason  we  feel  it  is  very  important  that 
covariance  adjusted  treatment  effects  be  the  basic  measure  of  effective- 
ness used  in  the  study.     Depending  on  the  grade  level  and  class  structure, 
some  restrictions  on  the  level  of  randomization  were  implemented.    We  will 
briefly  describe  the  experimental  designs  along  with  the  regression 
analyses  used  for  these  designs  for  each  one-year  study  for  a  given 
grade  level.    Discussions  of  the  results  are  followed  by  tables  summarizing 
all  the  treatment  effects  from  the  regression  analyses,  as  well  as  the 
posttest  means  by  treatment  group. 

The  Grade  1  Study 

The  Design  of  the  Experiment  in  Grade  1>  Year  2 

In  the  first  grade  the  only  available  CCC  CAI  curriculum  is  mathematics 
Primarily  because  of  this,  the  assignment  to  CAI  mathematics  curriculum  in 
grade  1,  year  2  was  on  a  classroom-by"classroom  basis.    Thus  an  entire  class 
-  room  of  first  graders  either  went  to  the  CAI  room  for  10  minutes  a  day  or 


never  went  there.    The  assignment  of  classrooms  to  CAI  was  done  randomly. 
Thus,  the  randomization  is  within  the  school  but  not  within  each  classroom. 
The  random  assignment  also  restricted  pairs  of  classrooms  in  the  same 
school,  if  possible,  to  achieve  balance  between  treatment.     Because  the 
CAI  assignment  was  not  completely  random  by  students,  classroom  differences 
.are  not  controlled  for  CAI  treatment'  effect  as  they  are  in  most  of  the       -  , 
other  one  year  studies  where  assignment  could  be  randomized  within 
class.  ;  • 

Regression  analyses  were  run  using  all  students  who  had  both  pretest  and 
post  test  data.    The  covariates  were  CST  math  subscores  and  CTBS  math  raw 
scores.     Indicator  variables  for  sex,  ethnicity,  and  school  were  also  used  to 
adjust  the  CAI  treatment  effect. 

Results  for  Grade  1,  Year  2 
I   *  

Table  4-4  gives  the  results  for  grade  1,  yiear  2  for  CST  and  CTBS 
mathematics  scores.     The  treatment  effect  for  the  CST  mathematics  total 
is  statistically  significant.     Students  with  mathematics  CAI  answered  16 
more  questions  correctly,  on  the  average,  than  did  students  without  CAI. 
The  two  parts  of  the  CST  contributed  about  equally  to  the  total  treatment 
effect.    For  CTBS  mathematics  scores,  the  treatment  effect  for  mathematics 
total  is  positive  but  not  significant  for  the  students  with  CAI.  Both 
subtest  scores  are  also  positive,  but  only  the  concepts  and  applications 
subtest  achieved  statistical  significance.  * 


-29- 


Table  4-4 

One-Year  Analysis  for  GRADE  -  1  YEAR  -  2. 

Posttest-  CST  AND  CTBS         Subject  -  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -  1-4. 


CST 


Treatment 
Group  . 


N 


Treatment 
Effect 


(t) 
119df 


Posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


PART  A 


M 
0 


49  g.50         6.34         30.14  9.71 

74  -  .23.39  13.20 


7.13 


1.19 


PART  B 


M 
0 


49 

•  74 


7.94 


4.95 


25.47 
18.93 


11.10 
13.17 


8.54 


0.93 


MATH 
TOTAL 


M 
0 


49 
74 


16.44 


6.18 


55.61 
42.32 


19.18 
25.77 


14.15 


1.16 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
119df 


Posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


COMPUTATION 


M 
0 


52 
78 


0.72  0.64 


16.19 
15.99 


8.19 
8.02 


6.22 


0.12 


CONCEPTS  AND  M 
APPLICATIONS  0 


52 
78 


1.32  2.06 


14.23 
13.50 


4.08 
5.53 


3.54 


0.37 


MATH 
TOTAL 


M 
0 


52 
78 


2.04  1.39 


30.42 
29.49 


11.08 
12.31 


8.13 


0.25 


2  1 J 
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The  Grade  2  Studies 

For  grade  2,  the  only  curriculum  available  was  mathematics  as  in 

grade  1.     Two  different  studies  were  conducted  in  years  1  and  3  to  provide 

within-CAI  comparisons.  The  situation  for  grade  2  across  the  fouk  years  of 
the  study  is  sumoomrized  as  follows: 

Year  1  Year  2  Year  3  Year  4 


CAI:  Cohort  CAI:M  (F  vs.  V) 

Grade  2  MM,  M  Controls  and  Controls 

Comparisons    Comparisons  Comparisons  


The  Design  and  Analyses  of  the  Two  Grade  2  Studies 

The  two  levels  of  treatment  which  were  assigifed  at  grade  2  in  year  1 
were  one  session  of  mathematics  every  day  (M)  and  two  sessions  of  mathema^ 
tics  every  day  (MM).     Second  gra4ers  were  randomly  assigned  i^the 

treatment  conditions  within  classrooms  in  the  first  year  of  the^study. 

^y 

Students  in  mixed  classes  (either  1-2-  or  2-3)  were  assigned  to  MM  for  the 
convenience  of  the  teachers.    This  design  was  chosen  to  measure  the 
effects  of  different  amounts  of  exposure  to  the  CAI  mathematics  curriculum 
for  grade  2  students.     The  within-classroom  randomization  to  different 
amounts  of  the  same  CAI  curriculum  was  difficult  to  implement  for  various 
practical  reasons. 

The  longer  period  of  CAI  time  for  the  MM  group  gave  teachers  an 
opportunity  to  work  with  students  remaining  in  the  classroom;  the  shorter 
period  of  time  for  the  M  group  di^^  not.     These  treatment  conditions  were 
not  received  well  by'  teachers  and  were  not  repeated  again  in  the  study 
after  year  1.  ■  ^ 


ERJC 
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Iri  year  3,  the  choice  of  curriculum  in  grade  2  was  again  restricted 
to  math,  but  instead  of  random  assignment  within  classroom,  students  were 
assigned  randomly  by  classroom  as  they  had  been  in  grade  !•     In  addition, 
within  each  classroom  assigned  to  receive  mathematics  CAI,  the  students 
were  randomly  divided  into  those  to  receive  the  "fixed"  strands  method  of 
delivery  of  the  curriculum  or  the  "variable"  strands  method  of  delivery. 
In  all^  other  instances  in  the  study,  mathematics  variable  strands 
were  ilsed.     In  variable  s.trands  questions  from  each  of  the  available 
strands  are  delivered  to  students  in  variable  order.     In  fixed  strands, 
one  can  amend  the  usual  procedure  and  concentifate  on  one  specific  strand — 
e.g.,  vertical  addition — and  receive  10  minutes  of  drill  and  practice  in 
that  strand  only.     Students  assigned  to  fixed  strands  received  fixed 
strands  for  only  part  of  their  CAI  lab  time,  usually  Tuesdays  and  Thursdays, 
while  receiving  variable  strands  during  the  rest  of  their  lab  time. 
Teachers  were  permitted  to  select  the  f ixed--mathematics  assignments  for 
their  students.    When  teachers  did  not  wish  to  make  the  selection,  CAI 
coordinators  did  so,  usually  by  assigning  fixed  strands  in  whichever  of 
the  various  strands  the  student's  placement  was  lowest. 
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Results  for  Grade  2 

Year  I.  In  the  first  year  of  the  study  the  CAI  labs  were  opened  in 
February.  The  CSTs  were  administered  in  June  after  3-1/2  months  of  CAI, 
but  the  CTBSs  were  administered  in  April  after  only  6-8  weeks  of  CAI. 

Table  4-5  gives  the  results  for  the  CST  and  CTBS  mathematics  scores. 
The  treatment  effects  for  the  total  CST  and  the  part  scores  are  not 
statistically  significant.    Students  who  received  2  sessions  of  CAI  in 
the  second  grade  did  only  marginally  better  than  students  taking  one 
session. 

For  the  CTBS  mathematics  scores,  the  treatment  effect  for  the 
mathematics  total  is  statistically  significant  as  are  the  treatment 
effects  for  both  computation  and  concepts  and  applications.     Students  who 
received  two  sessions  of  math  for  the  6-8  weeks  prior  to  CTBS  testing 
performed  better  than  students  receiving  one  session. 
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Table  4-5 

One-Year  Analysis  for  GRADE  -  2  YEAR  -  I. 

Posttest-  CST  AND  CTBS         Subject  -  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group. 
Schools  ■  1-4. 


CST 


Treatment  Treatment  (t) 

Group  N  Effect        197df . 


P^osttest  Residual  Standardized 

X  SD  SD  T.E. 


PART  A 


MM 
M 


155 
65 


0.49 


0.34 


34.87 
33.98 


12.23 
12.98 


8.30 


0.06 


PART  B 


MM 
M 


155 

65 


0.68 


0.48 


22.14 
23.74 


12.81 
13.13 


8.06 


0.08 


MATH 
TOTAL 


MM 
M 


155 
65 


1.17 


0.47 


57.01 
57.72 


23.69 
24.91 


14.32 


0.08 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
207df . 


_Posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


COMPUTATION 


MM 
M 


164 
66 


1.55 


2.07 


15.24 
15.11 


6.91 
6.96 


4.46 


0.35 


CONCEPTS  AND  MM 
APPLICATIONS  M 


164 
66 


1.14 


1.98 


15.20 
14'.  76 


5.23 
5.36 


3.43 


0.33 


MATH 
TOTAL 


MM 

M 


164 
66 


2.69 


2.38 


30.44 
29.87 


11.36 
11.65 
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Year  3.    We  shall  first  look  at  the  results  of  a  comparison  of 
CAI  students  with  students  who  did  not  receive  CAI.     Table  4-6  shQWs  the 
results  for  the  CSJ  and  CTBS  mathematics  scores.    Treatment  effects  for 
the  CST  total  score  and  the  part  scores  are  all  statistically  significant. 
Students  with  CAI  answered  12  more  questions  correctly  on  the  average 
than  did  students  without  CAI*     Expressed  in  proportions  of  the  residual 
standard  deviation,  the  adjusted  mean  for  the  CST  mathematics  total  for 
CAI  students  is  .79  of  a  standard  deviation  above  the  adjusted  mean  of 
non-CAI  students • 

For  the  CTBS  mathematics  scores,  the  treatment  effects  for  the  total 
and  the  subtests  are  all  statistically  significant.     The  CAI  students 
correctly  answered  2.55  more  computation  questions  and  1.08  more  application 
questions  than  did  students  without  CAI.    Expressed  in  a  proportion  of 
the  residual  standard  deviation,  the  adjusted  mean  for  CTBS  mathematics 
computation  for  CAI  students  is  .31  of  a  standard  deviation  above  the 
adjusted  mean  of  non'^CAI  students. 
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Table  4-6 

One-Year  Analysis  for  GRADE  -'2  YEAR  -  3. 

Posttest  -  CST  AND  CTBS         Subject  -  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -  1-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t). 
189df . 


posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


PART  A 


M 
0 


100 
99 


5.72 


4.44 


39.07 
31.39 


9.91 
12.48 


8.75 


0.65 


PART  B 


M 
0 


100 
99 


6.37 


5.21 


27.29 
18.14 


12.88 
11.42 


8.30 


0.77 


MATH 
TOTAL 


M 
0 


100 
99 


12.08 


5.35 


66.36 
49.54 


21.36 
22.72 


15.34 


0.79 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
196  df. 


posttest, 
X  SD 


Residual  standardized 
SD  T.E. 


COMPUTATION 


M 
0 


104 
102 


2.55 


3.37 


18.31 
14.26 


6.29 
6.90 


5.03 


0.51 


CONCEPTS  AND  M 
APPLICATIONS  0 


104 
102 


1.08 


2.10 


16.29 
13.71 


-4.78 
5.38 


3.43 


0.31 


MATH 
TOTAL 


M 
0 


104 
102 


3.62 


3.41 


34.60 
27.96 


10.11 
11.20 


7.08 


0.51 
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Students  in  second-grade  classrooms  assigned  to  CAI  in  Year  3 
were  randomly  assigned  to  receive—or  not  receive — part  of  their  instruc- 
tion in  the  fixed-strands  mode.    Ordinaxrily  students  receiving  mathematics 
CAI  took  it  in  the  variable  mode;  i.e.,  questions  from  all  available 
components  of  the  mathematics  curriculum  appeared  in  random  sequence.  A 
question  on  measurement  might  be  followed  by  one  on  vertical  addition 
followed  by  one  on  horizontal  subtraction.    In  the  fixed-strand  mode, 
students  were  assigned  for  part  of  their  CAI  time  to  a  specific  strand — 
e.g.,  horizontal  addition  selected  by  the  teacher. 

Table  4-7  shows  the  results  for  the  CST  and  CTBS  mathematics  scores. 
None  of  the  treatment  effects  is  statistically  significant,  although  there 
appears  to  be  a  trend  favoring  students  who  took  part  of  their  instruction 
in  fixed  strands. 
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Table  4-7 

One-Year  Analysis  for  GRADE  -  2  YEAR 
Posttest  «  CST  AND  CTBS 


3. 


Subject  «  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group, 
\    School^  "1-4. 


CSTi 


Treatment 
Group 


Treatment        (t)             Posttest  Residual  Standardized 

Effect        90df.  X   SD  SD  t.E. 


PAkT  A 


M-VARIABLE 
M-FIXED 


52 
48 


-2.80  -1.62 


38.00 
40.23 


11.25 
8.05 


8.00 


-0.35 


PART  B 


M-VARIABLE 
M-FIXED 


52 
48 


0.11 


0.07 


28.14 
26.38 


13.59 
11.99 


8.01 


-0.01 


MATH 
•^loTAL 


M-VARIABLE 
M-FIXED 


52 
48 


-2.69  -0.91 


66.14 
66.61 


23.32 
19.00 


14.37 


-0.19 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
94df 


posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


COMPUTATION    M-VARIABLE  55 
M-FIXED  49 


-0.37  -0.39 


17.91 
18.76 


6.88 
5.52 


4.56 


-0.08 


CONCEPTS  AND  M-VARIABLE  55 
APPLICATION    M-FIXED  49 


-0.91  -1.44 


15.73 
16.92 


4.89 
4.58 


3.03 


-0.30 


MATH 
TOTAL 


M-VARIABLE 
M-FIXED 


55 
49 


-1.27  -0.95 


33.64 
35.67 


10.68 
9.30 


6.45 


-0.20 
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Summary  of  Grade  2  Data 

In  only  one  case  were  CAI  students  compared  with  non-CAI  students  In 
grade  2.     In  year  3,  6econd-grade  students  who  received  mathematics  CAI 
performed  significantly  better  than  students  without  CAI«  ^' 

When  students  were  assigned  to  one  or  two  sessions  of  mathematic's^ 
CAI,  those  with  two  sessions  performed  significantly  better  on  the  CTBS 
mathematics  subtests  than  those  assigned  to  one  session* 

Where  students  were  assigned  fixed  strands  for  part  of  their  CAI ^ 
lab  time,  no  statistically  significant  differences  in  treatment  effects 
occur,  although  the  trend  seems  to  favor  the  fixed-strand  group* 
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The  Grade  3  Studies 

The  third-grade  studies  and  curriculums  compared  are  summarized  as 
follows : 

Year  1  Year  2   Year  3  Year  4 

Cohort  CAI:  Cohort  .CAI: 

Grade  3  Controls  MM,  ML  Controls  M,  L 

Comparisons       Comparisons      Comparisons  2  

For  grade  3,  CAI  curriculums  were  administered  in  year  2  and  year  A. 

The  other  two  years  provided  the  cohort  cbntrols  and  comparison  school 

controls  which  are  discussed  later  in/the  report. 

The  Design  of  the  Experiment  in  Grade  3,  Year  2  and  Year  4 

In  the  third  grade,  two  CAI  curriculums  were  aval  lab  le-*-mat  hematics 
and  language.  *  la  year  2,  students  were  randomly  assigned  within  classrooms 
either  to  20  minutes  of  mathematics  CAI  a  day  (MM)  or  to  10  minutes  of 
mathematics  and  10  minutes  of- language  CAI  (ML).     At  that  time,  the 
language  curriculum  was  not  more  extensively  utilized  because  of  the 
'uncertainties  as  to  whether  or  not  it  could  be  used  by  students  in  grade 
3.     It  turned  out  that  10  minutes  a  day  of  CAI  language  curriculum  could 
be  implemented  in  a  satisfactory  way  in  grade  3.     With  this  experience, 
in  year  4  students  were  randomly  assigned  to  10  minutes  of  mathematics  a 
day  (M)  or  10  minutes  of  language  a  day.  (L).'   This  design  gives  us  direct  . 
comparison  between  the  mathematics  curriculum  and  the  language  curriculum. 
In  both  designs  students  were  assigned  to  either  treatment  on  a  student-by- 
student  basis  rather  than  a  class-by-class  basis.    Thus,  in  a  typical 
third-grade  CAI  .class,  there  will  be  approximiteiy  equal  numbers  of 
students  in  either  condition. 


Results  for  Grade  3 

Year  2,    Table  4-8  gives  the  results  for  grade  3,-  year  2  for 
;CST  and  CTBS  mathematics  scores.    The  treatment  effects  for  the  GST  total 
and  part  scores  are  statistically  significant*     Students  who  received  2 
sessions  of  mathema|:ics  GAI  daily  performed  significantly  better  on  the 
mathematics  GST  than  students  assigned  to  mathematics  and  language  GAI* 
In  terms  o^f  the  residual  standard  deviation,  the  adjusted  means  oi^  the  MM 
students  are  about  two^thirds  of  a  standard  deviation  higher  than  the 
adjusted  means  of  ML  students  on  the  GST  total  mathematics  score* 

Only  one  of  the  GTBS  subtests  shows  statistically  significant 
treatment  effects*     Students  assigned  MM  did  significantly  better  on 
mathematics  computation  than  did  students  assigne'd  to  ML*     The  MM  students 
are  about  one^third  of  a  standard  deviation  better  than  the  ML  students 
on  mathematics  computation* 
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Table  4-8 

One-Year  Analysis  for  GRADE  -3  YEAR  -  2. 

Posttest-  CST  AND  CTBS  '  Subject  -  MATHEMATICS. 

Treatqent  effects  and  posttest  means  by  treatment  group, 
Schools  -  1-4. 


CST 


treatment 
Group 


Treatment  (t) 
Effect  173df. 


Posttest 
X  SD 


Residual  Standardized. 
,    SD    .  T/E, 


PART  A 


MM 
ML 


111 

80' 


4.56 


3.15 


44.22 
39.80 


11.74 
14.31 


9.20 


0.50 


PART  B 


MM 
ML 


111 
80 


5. '60  4.69 


34.94  12.33 
29.21  13.70 


7.60 


0.74 


MATH 
TOTAL 


CTBS 


MM 
ML 


Treatment 
Group 


111 
80 


10.16 


Treatment 
Effect' 


4.29 


(t) 
186df . 


79.15 
69.01 


22.91 
26.73 


Posttest 


SD 


15.07 


0.67 


Residual  Standardized 
SD  T.E. 


COMPUTATION          MM  113 
ML  •  91 


2.80 


2.44 


30.50 
28.01 


11.16 
11.63 


7. '4  4 


0.38 


CONCEPTS 


MM 
ML 


113 
91 


-0.14     •  -0.22 


12.79 
12.91 


6.47 
6.51 


4.04 


-0.03 


APPLICATIONS 


MM 
ML 


113 
»91 


0.15 


0.23 


10.82 
11.09 


6.34 
6.52 


4.17 


0.04 


MATH 
TOTAL 


MM 
M 


113 
91 


2.Tb1 


1.47 


•54.11 
52.01 


21.51 
22.89 


12.35 


0.23 


2v.  ) 
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Table  4-9 

One-Year  An,aly sis  for  GRADE  -  3  YEAR  -  2.  , 

Post test-  CST  Subject  -  READING  AND  LANGUAGE. 

Schools  Posttest  means  by  treatment  group. 


Treatment 
Subtest     a  Group 


READING 


LANGUAGE 


TOTAL 


MM 
ML 


MM 
ML 


MM 
ML 


115 
78 


115 
78 


115 
78 


Treatment  (t) 
Effect  i75df 


-1.64 


3.96 


2.32 


0.98 


2.90 


0.85 


^osttest 

X,  SD 


36.65 
36.83 


31.51 
37.09 


68.16 
73.92 


15.40 
15.31 


13.73 
13.21 


Residual  Standardized 
SD  T.E. 


10.65 


8.70 


27.37  17.35 
26.99  ■ 


-0.15 


0.46 


0.13 


Tables  4-9  and  4-10  present  the  reading  and  language  results  for 
the  CST  and  CTBS  tests.    The  CST  results  reported  in  Table  4-9  are  based 
on  a  reading  test  and  a  language  test.     It  was  not  expected  a  priori  that 
ML  students  would  perform  better  on  the  reading  portion  of  the  CST,  but  , 
their  performance  on  the  language  portion  of  the  test  was  expected  to  be 
better.    Although  the  results  for  reading  are  not  significant,  the 
results  for  language  are  statistically  significant.     Students  assigned 
to  ML  answered,  on  the  average,  four  more  questions  correctly  than  MM 
students.    The  adjusted  mean  score  for  ML  students  on  the  language  CST 
is  .46  of  a  standard  deviation  higher  than,  the  adjusted  mean  score  for 
MM  students . 

.  For  the  CTBS  data,  none  of  the  treatment  effects  is  statistically 
significant.  ^  , 
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Table  4-10 

One-Year  Analysis  for  GRADE  -  3  YEAR  =  2. 

Posttest  -  CTBS 


Subject  -  READING  AND  LANGUAGE. 


Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  =1-4. 


CTBS  Treatment  Treatment  (t) 

READING  Group  N  Effect        ISOdf . 


posttest  Residual  Standardized 

X   SD  SD  T.E. 


VOCABULARY 


MM 
ML 


111 
89 


-0.59  -1.02 


16.65 
16.36 


8.81 
8.97 


3.75 


-0.16 


COMPREHENSION  MM 
ML 


111 
89 


-1.56  -1.89 


18.28 
17.20 


10.22, 
11.26 


5.35 


-0.29 


READING 
TOTAL 


MM 
ML 


111 
89 


-2.15  -1.79 


34.94 
33.56 


18.48 
19.60 


7.77 


-0.28 


CTBS 

LANGUAGE 


Treatment 
Group 


Treatment        (t)             Posttest  Residual  Standardized 

Effect        179df  X        _SD  SD  ^.E. 


SPELLING 


MM 
ML 


111 
88 


-0.75  -0.77 


25.51 
25.02 


8.35 
8.47 


6.28 


-0.12 


MECHANICS 


MM 
ML 


111 
88 


0.26 


0.66 


8.56 
9.28 


4.40 
4.68 


2.58 


0.10 


EXPRESSION 


MM 
ML 


111 
88 


0.02 


0.04 


13.54 
13.77 


7.35 
7.45 


3.34 


0.01 


LANGUAGE 
TOTAL 


MM 
ML 


111 
88 


-0i46  -0.34 


47.61 
48.08 


17.84 
18.57 


8.87 


0.05 
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Year  A,     Table  A-11  presents  the  results  for  grade  3,  year  A 
CST  and  CTBS  mathematics  tests.     The  treatment  effects  for  the  CST 
total  and  part  scores  are  all  statistically  significant.  Students 
assigned  to  mathematics  CAI  completed  an  average  of  12, 78  more  questions 
correctly  than  students  assigned  to  language  CAI.    The  adjusted  mean 
score  for  the  M  group  is  .82  of  a  standard  deviation  higher  than  the 
adjusted  mean  score  for  the  L  group. 

Ncvne  of  the  CTBS  tests  demonstrate  significant  treatment  effects. 
Results  of  the  CTBS  computation  test,  with  a  treatment  effect  of  2.09 
items,  approaches  significance. 


I 
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Table  4-11 

One-Year  Analysis  for  GRADE  »  3  YEAR  -  4. 

Posttest-  CST  AND  CTBS  .  Subject  -  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -  1-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
188df 


JPosttest 
X  SD 


Residual  Standardized  * 
SD  T.E.  *; 


PART  A 


M 
L 


124 
82 


6.97 


5.04 


42.39 
37.83 


11.71 
14.77 


8.89 


0.78 


PART  B 


M 
L 


124. 

82 


5.80 


4.45 


29.50 
26.97 


13.54 
13.15 


8.40 


0.69 


MATH 
TOTAL 


M 
L 


124 
82 


12.78 


5.30 


71.89 
64.80 


24.05 
26.85 


15.52 


0.82 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
148df 


JPosttest 
X  SD 


Residual  Standardized 
SD  T.E. 


COMPUTATION  M 
L 


90 
77 


2.09 


1.80 


30. 09 
28.22 


11.39' 
11.31 


6.95 


0.30 


CONCEPTS 


M 
L 


90 
77 


-0.67 


-1.23 


13.61 
14.34 


5.76 
5.97 


3.28 


-0.20 


APPLICATIONS 


MATH 
TOTAL 


M 
L 

M 
L 


90 
77 

90 

77 


0.06 


1.47 


0.10  11.55 
11.44 


5.98  3.86 
6.31  ' 


0.81        55.25        21.19  10.91 
54.00  21.65 


0.02 


0.13 
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Tables  A-12  and  4-13  present  the  reading  and  language  results  for 
grade  3,  year  4.     Results  for  the  reading  and  language  CST  scores  look 
similar  to  those  reported  for  grade  3,  year  2.    No  significant  effect 
appears  for  the  reading  portion  of  the  CST,  but  a  statistically  significant 
effect  shows  up  for  the  language  portion.     The  adjusted  mean  for  the 
language  students  is  .41  of  a  standard  deviation  higlier  than  the  adjusted 
mean  for  students  in  the  mathematics  CAI  curriculum. 

For  the  CTBS  reading  and  language  scores  (Table  4-13),  statisti- 
cally significant  treatment  effects  are  found  for  vocabulary,  reading 
total,  spelling  and  language  total.    For  those  four  test  scores  the 
adjusted  means  for  language  CAS  students  are  one-third  to  one-half  of  a 
standard  deviation  higher  than  the  adjusted  means  for  mathematics  CAI 
students . 

Table  A-12 

One-Year  Analysis  for  GRADE  -  3  YEAR  =  4.  ' 

Posttest-  CST  •  Subject  -  READING  AND  LANGUAGE. 

Treatment  effects  and  posttest  means  by  treatment,  group, 
Schools  "  1-4. 


Subtest 


Treatment 
Group 


Treatment 
Effect 


(t) 
156df 


posttest 

X  SD 


Residual  standardized 
SD  T.E. 


READING  M 
L 


93 
82 


1.80 


0.95 


40.40 
44.56 


15.69 
14.44 


11.90 


0.15 


LANGUAGE 


M 
L 


93 
82 


3.90 


2.61 


37.41 
42.86 


12.71 
12.68 


9.40 


0.41 


TOTAL 


M 
L 


93 
82 


5.70 


1.82. 


77.81 
87.39 


27.06 
25.87 


19.70 


0.30 
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Table  4-13 

One-Year  Analysis  for  GRADE  -  3  YEAR  -  4. 

Posttest-  CTBS  Subject  -  READING  AND  LANGUAGE. 

Treatment  effects  and  posttest  means  by  treatment  group. 
Schools  ■  l'^4. 


CTBS 
READING 


Treatment 
Group 


Treatment 
Effect 


(t) 
155df 


posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


VOCABULARY  M 
L 


93 
82 


1.64 


2.15 


16.99 
20.15 


8.43 
8.91 


4.81 


0.34 


COMPREHENSION 


M 
L 


93 
82 


1.59 


1.70 


18.35 
2K.57 


10.75 
10.84 


5.88 


0.27 


READING 
TOTAL 


M 
L 


93 
82 


3.23 


2.16 


35.34 
41.72 


18.44 
19.13 


9.42 


0.34 


CTBS 
LANGUAGE 


Treatment 
Group 


Treatment        (t)  posttest  Residual  Standardized 

Effect        155df  X  SD  SD  T.E. 


SPELLING 


LANGUAGE 
MECHANICS 


M 
L 


M 
L 


93 
82 


93 
82 


2.64 


0.90 


3.20 


1.92 


•25.34 
29.76 


9.35 
10.94 


8.53 
7.98 


4.15 
5.01 


'5.20 


2.92i 


0.51 


0.31 


LANGUAGE 
EXPRESSION 


M 
L 


93 
82 


0.50 


0.78 


14.54 
16.18 


6.il 

64  85 


4.05 


0.12 


LANGUAGE 
TOTAL 


M 
L 


93 
82 


4.04 


2.93 


49.24  16.89  8.67 
56.88  17.73 


0.47 
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Summary  of  the  Grade  3  Data 

Overall,  students  in  grade  3  who  received  mathematics  CAI  performed 
significantly  better  on  mathematics  CSTs  and  on  CTBS  computation  subtests 
than  students  receiving  language.     Conversely,  those  students  assigned  to 
the  language  CAI  curriculum  performed  significantly  better  on  the  language 
portion  of  the  CST.    The  CTBS  language  results  differ  from  year  2  to 
year  4.     In  year  2  the  treatment  effects  for  the  spelling  subtest  and 
language  total  are  both  negative;  in  year  4  they  are  both  positive  and 
both  statistically  significant. 


i 
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The  Grade  4  Studies 

The  one-year  studies  for  grade  4  and  curriculums  compared  are  sum- 
marized as  follows; 

Year  1  Year  2  Year  3  Year  4 

CAI;  Cohort  CAI:  CAI; 

Grade  4  MM,  MRL,  RL       Controls  MM,  RL  M,R,L,C 

Comparisons       Comparisons  Comparisons  

Because  two  longitudinal  studies  span  fourth  grade  and  a  special  study 
occurred  in  year  4,  we  have  three  studies  of  CAI  for  grade  4, 

The  Design  and  Analyses  for  Grade  4 

In  year  1,  all  fourth-grade  students  were  given  two  10-minute 

sessions  on  the  computer  with  three  different  combinations.    The  three 

CAI  curriculum  combinations  are  MM  (two  sessions  of  math)  RL  (one 

session  of  reading  and  one  session  of  language),  and  MR/L  (one  session  of 

math  followed  by  one  session  of  reading  or  language  alternatively). 

Students  were  randomly  assigned  to  one  of  the  three  treatment  curriculums. 

* 

Comparisons  across  treatments  represent  the  randomized  part  of  the  research 
design.     There  are  three  levels  of  math  treatments:     students  in  MM  are 
the  high  math  group;  students  in  MRL  are  the  low  math  group;  and  students 
in  RL  are  the  zero  math  group.     Similarly,  there  are  three  levels  of  the 
R  and  L  curriculums  with  RL  representing  the  high  group,  MRL  the  low, 
and  MM  the  zero  condition. 
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We  made  the  assumption  (based  on  a  lack  of  evidence)  that  the 
Reading  and  Language  CAI  currlculums  were  not  mutually  exclusive.  I.e., 
they  covered  much. of  the  same  material  In  a  similar  format.  Therefore, 
equal  time  was  given  to  math  skills  and  v,erbal  skills. 

The  design  gave  us  a  randomized  comparison  among  three  levels  of 
math  and  verbal  curriculum* 

In  year  3,  the  students  could  be  classified  Into  two  groups  according 
to  their  experience  with  CAI.     Because  of  the  LAUSD  desegregation  plan, 
there  were  six  classrooms  of  students  In  grade  4  at  School  1.    Most  of 
these  students  were  new  to  CAI.     The  grade  4, students  In  the.  other  three 
CAI  schools  had  been  In  the  CAI  program  for  two  previous  years.     The  two 
currlculums  available  were  MM,  which  represented  two  sessions  of  10 
minutes  of  math,  and  RL,  which  represented  one  session  of  reading  and  one 
session  of  language.     Within  each  classroom,  students  who  were  new  to  CAI 
were  randomly  assigned  to  either  MM  or  RL.     Students  who  were  previously 
In  CAI  would  receive  the  same  curriculum  as  they  received  In  year  2. 
This  was  a  section  of  the  grade  2  to  grade  3  longitudinal  study.  Because 
the  small  CAI  lab  at  School  1  could  not  cope  with  that  many  students — 
about  130 — on  a  dally  basis,  the  grade  4  classrooms  were  assigned 
randomly  to  the  CAI  lab  either  on  Monday,  Wednesday  and  Friday  (MWF),  or 
Tuesday  and  Thursday  (T/Th).     Students  In  the  MWF  group  were  to  have 
twenty  more  minutes  CAI  than  those  In  T/Th  group  weekly. 

In  year  4,  the  one-year  study  for  grade  4  was  designed  to  determine 
the  separate  contributions  of  four  CAI  currlculums:    mathematics  (M),  language 
(L),  reading  (R),  and  a  new  reading  for  comprehension  (C)  curriculum.  In 


School  1,  approximately  150  fourth-grade  students  were  randomly  assigned 
to  M,  R  or  C.     In  the  other  CAI  schools  fourth-grade  students  were 
randomly  assigned  to  M,R,  or  L.    All  students  were  new  to  CAI  and  went  to 
the  CAI  lab  for  10  minutes  of  CAI  daily  from  mid-October  to  the  end  of  May. 
This  was  the  first  time  fourth-grade  students  received  CAI  on  a  daily 
basis  over  the  whole  academic  year.     This  study  was  designed  to  measure 
specifically  the  one-year  treatment  effect.     Students  in  School  1  took 
mathematics,  reading,  and  comprehension  CST  posttests  while  students  in 
other  CAI  schools  had  mathematics,  reading  and  language  CST  posttests. 
Results  appear  on  the  following  pages. 
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Results  for  Grade  4 

Year  1»     Table  4-14  presents  the  mathematics  results  for  fourth 
graders  in  year  1.     For  the  CST  data,  i;he  treatment  effects  for  the  MM 
group  are  all  statistically  significant  and  for  the  MRL  group  are  all 
positive  but  not  significant.     The  MM  group  is  .9  of  a  standard  deviation 
higher  than  the  RL  group  on  the  CST  mathematics  total.     For  the  CTBS 
data,  all  the  treatment  effects  are  negative,  and  those  for  the  MRL  group 
in  computation  and  mathematics  total  are  statistically  significant. 

Table  4-15  presents  the  reading  results.    For  the  CST  reading 
total,  the  treatment  effect  for  the  RL  group  is  statistically  significant. 
The  RL  group  is  .47  of  a  standard  deviation  above  the  MM  group  on  the  CST 
reading  total.     For  the  CTBS  data,  there  are  statistically  significant 
treatment  effects  for  the  RL  group  in  reading  vocabulary  and  reading 
total,  with  the  performance  of  the  RL  group  more  than  one-third  of 
a  standard  deviation  above  that  of  the  MM  group. 

Table  4-16  presents  the  language  results  for  grade  4,  year  1.  For 
the  CST  language  total,  the  treatment  effects  for  both  the  RL  and  MRL 
groups  are  statistically  significant,  with  the  performance  of  the  RL  group 
more  than  a  standard  deviation  above  that  of  the  MM  group.    For  the  CTBS 
data,  statistically  significant  treatment  effects  are  found  for  the  RL 
group  in  spelling  and  language  total. 
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Table  4-14  ' 

One-Year  Analysis,  for  GRADE  -  4  YEAR  -  i. 

Posttest  -  CST  AND  CTBS  Subject  -  MATHEMATICS. 

Treatment  effects  atid  posttest  means  by  treatment  group, 
Schools  "  1-6. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
158df . 


P^osttest 
X  SD 


Residual  Standardized 
SD  T.E. 


PART  A 


MM 

MRL 

RL 


52 
66 
57 


9.29 
3.06 


4.98 
1.73 


35.94 
29.76 
27.35 


14.65 

ao.33 

11.79 


9.38 


0.99 
0.33 


PART  B 


MM_ 
MRT 
RL 


52 
66 
57 


5.23 
2.19 


3.25 
1.44 


23.92 
20.29 
18.75 


12.39 
10.33 
9.80 


8.08 


0.65 
0.27 


MATH 
TOTAL  , 


MM 

MRL 

RL 


52 
66 
57 


14.51 
5,25 


4.52 
1.73 


59.86 
50.04 
46.10 


25.44 
23.04 
20.41 


16.12 


0.99 
0.33 


CTBS 


Treatment  Treatment  (t) 

Group  N  Effect  180df 


posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


COMPUTATION 


MM 

MRL 

RL 


63 
67 
68 


-0.11 
-2.19 


■0.07 
-1.48 


35.57 
33.16 
35.71 


11.18 
11.11 
10.57 


8.14 


-0.01 
-0.27 


CONCEPTS 


MM 

MRL 

RL 


63 

67 
68 


-1.24 
-0.95 


■1.81 
■1.40 


15.90 
15.94 
17.44 


6.47 
5.48 
.5.53 


3.73 


-0.33 
-0.25 


APPLICATIONS 


MM 

MRL 

RL 


63 

67 
68 


■1.00 
-1.35 


-1.25 
-1.71 


13.33 
'1'2.'79 
15.09 


7.16 
6.60 
6.29 


4.36 


-0.23 
-0.  31 


MATH 
TOTAL 


MM 

MRL 

RL 


63 

67 
68 


-2.34 
-4.50 


-1.01 
■1.95 


64.60 
61.90 
68.24 


22.16 
20.82 
20.64 


12.69 


-0.18 
-0.35 


9  7  ^ 
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Table  ^-15 

One-Year  Analysis  for  GRADE  -4  YEAR  -  1. 

Posttest  -  CST  AND.CTBS  Subject  -  READING. 

i 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  ■  1-6. 


CST 


Treatment 
Group 


Treatment        (t)            Posttest  Residual  Standardized 

Effect        183df         __X    SD  SD  ^ 


PART  A 


MM 

MRL 

RL 


61 
76 
68 


1.98 
4.28 


1.35 
2.75 


45.70 
47.59 
51.85 


13.12 
9.78 
5.31 


8.34 


0.24 
'0.51 


PART  B 


MM 

MRL 

RL 


61 
76 
68 


-0.65 
2.66 


-0.48 
1.84 


30.39 
29,62 
35.40 


13.40 
11.84 
10.71 


7.78 


-0.08 
0.34 


READING 
TOTAL 


MM 

MRL 

RL 


61 
76 
68 


1.33 
6.94 


0.51 
2.53 


76.10 
77.21 
87.25 


25.32 
20.74 
14.85 


14.73 


0.09 
0.47 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
196df 


posttest 
X  SD 


Residual  Standardized 
T.E. 


VOCABULARY 


MM 

MRL 

RL 


62 
70 
73 


0.23 
1.43 


0.24- 
1.53 


2Z.-35 
21.96 
24.27 


9.05 
8.72 
9.28 


5.20 


0.04 
0.28 


COMPREHENSION 


MM 

MRL 

RL 


62 
70 
73 


-0.28 
1.59 


-0.22 
1.27 


25.18, 

24.33 

27.62 


19.89 
11.25 
11.50 


7.01 


■'J  04 
0.23 


READING 
TOTAL 


MM 

MRL 

RL 


62 
70 
73 


-0.05 
3.02 


-0.03 
1.58 


47.53 
46.29 
51.89 


19.89 
18.98 
19.42 


10.62 


0.00 
0.28 
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Table  4-16.  ••  ' 

One-Year  Analysis  for  GRADE  -  4  YEAR  -  l"^ 

Posttest  -  CST  AND  CTBS  Subject  -  LANGUAGE.' 

Treatment  effects  and. posttest  means  by  treatment  group,; 
Schools  *  l',-A. 


CST 


Treatment      "  Treatment  (t) 

Group  N  Effect  183df 


JPos'ttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


MM 

MRL 

RL 


61 
76 
68 


2.23 
6.43 


1.81 
4.93 


39.68 
41.71 
47.88 


.IK99 
10.51 
6.81 


7.00 


0.32 
0.92 


PART  B 


MM 

MRL 

RL 


61 
76. 
68 


3.  as 

7.35 


2.40 
5.46 


34.44 
37.05 
43.81 


12.09 
9.94 
8.55 


7.23 


0.,42 
1.02 


LANGUAGE 
TOTAL  ' 


MM  •    61  -  74.13  23.07 

MRL  76     -       5.28         2.37         78.76  19.21 

RL  ,68  13.77  5.83         91.96  14.27 


12.68 


0.42 
1.09 


CTBS 


Treatment 

Group      >   N 


Treatment  (t) 
Effect'  196df 


JPosttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


SPELLING 


MM  ' 
MRL 
RlJ 


62 
70 
73 


0. 54 
2.25 


0.46 
1.86 


28.71 
29.10 
31 . 62 


9.73 
8.04 
9.97 


6.71 


0.08 
0.34 


MECHANICS 


MM\  ■ 
MRL 
RL  \ 


62 

70. 
73 


-0.02 
0.69 


-0.03 
1.15 


10.69 
10.66 
11.68 


4.90 
4.86 
4.92 


3.34 


-0.01 
0.21 


EXPRESSION 


MM 

MRL 

RL 


62 
70 
73 


-0.34 
0.98 


-0.41 
1.18 


16.98 
16.36 
18.30 


7.42 

7.54 
7.97 


4.65 


-0.07 
0.21 


LANGUAGE 
TOTAL 


MM 

MRL 

RL 


62 
70 
73 


0.19 
3.92 


0.09 
1.94 


56.39 
5^.11 
61.60 


19.04 
18.06 
20.22 


11.24 


0.02 
0.'35 
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Year  3,    Table    4-17  presents  the  mathematics  results  for  fourth 
graders  in  Year  3.     For  the  CST  data,  treatment  effects  for  the  MM  group 
are  all  statistlcally^signif icant.    Performance  of  the  MMgroup  is  almost 
two-thirds  of  a  standard  deviation  above  that  of  the  RL  group  on  the  CST 
mathematics  total.     For  the  CTBS  data,  the  only  statistically  significant 
treatment  effect  is  for  the  MM  group  in  mathematics  computation. 

Table    4-18  presents  the  reading  results.    For  the  CST  reading  total 
as  well  as  part  B,  treatment  effects  for  the  RL  group  are  statistically 
significant.     Overall,  performance  of  the  RL  group  exceeds  that  of  the  MM 
group  by  one-third  of  a  standard  deviation.     For  the  CTBS  data,  the  RL 
group  again  out-performed  the  MM  group  with  statistically  significant 
treatment  effect^  in  comprehension  and  reading  total. 

•    Table  4-19  presents  the  language  results.    For  the  CST  data,  all 
treatment  effects  for  the  RL  group  are  statistically  significant,  with  the 
adjusted  mean  for  the  RL  group  about  two-thirds  of  a  standard  deviation 
above  that  of  the  MM  group.    For  the*CTBS  data,  statistically  significant 
treatment  effects  for  the  RL  group  occur  for  language  mechanics, 
language  expression  and  the  language  total.     Overal^l,  the  adjusted  mean 
for  RL  students  on  the  CTBS  language  total  is  .28  of  a  standard  deviation 
higher  than  that  of  the  MM  students. 
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Table  4-17 

One-Year  Analysis  for  GRADE  -  4  YEAR  -  3. 

Posttest-  CST  AND  CTBS  Subject  -  MATHEMATICS.- 

Treatment  effects  and  posttest  means  by  treatment  group. 
Schools  "1-4. 


CST 


Treatment 
Group 


Treatment  (t) 
.  Effect  256df 


P^osttest 
X  SD 


Residual 

'  SD 


Standardized 
T.E. 


PART  A 


RL 
MM 


112 
169 


5.17 


5.24 


35.78 
41.96 


11.32 
11.76 


7.80 


0.66 


PART  B 


RL 

MM 


112 
169 


3.24 


3.54 


24.89 
29.19 


10.42 
13.45 


7.24 


0.45 


MATH 
TOTAL 


RL 
MM 


112 
169 


8.41 


4.92 


60.67 
71.15 


20.64 
24.26 


13.51 


0.63 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
252df 


posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


COMPUTATION 


MM 
RL 


169 
107 


2.28 


2.15 


37.74 
36.52 


10.30 
9.68 


8.26 


0.28 


CONCEPTS 


MM 
RL 


169 
107 


0.23 


0.45 


16.95 
17.72 


5.95 
5.61 


3.94 


0.106 


APPLICATIONS 


MM 
RL 


169 
107 


0.41 


0.78 


14.64 
15.06 


6.:^8 
7.06 


4.17 


0.10 


MATH 
TOTAL 


MM 
RL 


169 
107 


2.92 


1.73 


69.34 
69.30 


20.22 
19.80 


13.14 


0.22 


ERIC 


^  (  I.) 


/ 
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Table  4-18 

One-Year'  Analysis  for  GRADE  ■  4  YEAR  «  3. 

Posttest-  CST  AND  CTBS  Subject  »  READING. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  "  1-4. 


CST 


Treatment  Treatment  (t) 

Group  N  Effect  ,  240df 


_Posttest 
X  SD 


Residual  Standardized 
SD   111:  


PART  A 


MM 
RL 


161 
106 


1.17 


38 


50.07 
52.11 


9.47 
6.32 


6.85 


0.17  = 


PART  B 


MM 
RL 


161 
106 


2.75  2.98 


34.76 
38.90 


12.45 
9.65 


7.01 


0.39 


READING 
TOTAL  . 


MM 
RL 


161 
.106 


3.92  2.57 


84.83 
91.02 


20.42 
14.96 


11.61 


0.34 


CTBS 


treatment 
Group 


Treatment 
Effect 


(t) 
245df 


_Posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


VOCABULARY 


MM 
RL 


165 
108 


1.43 


1.91 


22.73 
24.35 


10.11 
10.52 


5.73 


0.25 


COMPREHENSION  MM 
RL 


165 
108 


1.-97 


2.02 


26.09 
27.04 


12.09 
11.99 


7.48 


0.26 


READING 
TOTAL 


MM 
RL 


165 
108 


3.40 


2.22 


48.82 
51.39 


21.28 
21.32 


11.71 


0.29 


ERIC 
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Table  4-19 

One-^ear  Analysis  for  GRADE  -  4  YEAR  -=  3. 

/ 

Posttest-  CST  AND  CTBS  Subject  *  LANGUAGE. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  =  1-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
240df 


£osttest 
X  SD 


Residual  Standardized 
SD  T.E. 


^ART  A 


MM 
RL 


161 
106 


2.75 


3.43 


43.89 
48.16 


9.75 
6.71 


6.08 


0.45 


PART  B 


MM 
RL 


161 
106 


4.74 


5.18 


37.51 
43.33 


10.42 
8.18 


6.96 


0.68 


LANGUAGE 
TOTAL 


MM 
RL 


161 
106 


7.49 


4.96 


81.41 
91.49 


18.94 
13.94 


11.48 


0.65 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
245df 


posttest 
X  SD 


Residual  Standardfzed 
SD  T.E. 


SPELLING 


MM 
RL 


165 
108 


0.23 


0.27 


29.34 
29.36 


9.70 
9.45 


6.72 


0.03 


MECHANICS 


MM 
RL 


165 
108 


1.39 


3.16 


11.25 
12.52 


4.91 
4.40 


3.37 


0.41 


EXPRESSION 


MM 
RL 


165 
108 


1.57 


2.49 


17.90 
19.20 


7.48 
6.62 


4.83 


0.33 


LANGUAGE 
TOTAL 


MM 
RL 


165 
108 


3.20 


2.11 


58.50 
61.08 


19.94 
18.11 


11.62 


0.28 


ERIC 
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Year  4,    Table  4-20  presents  the  mathematics-  results  for  fourth 
graders  in  year  4,     For  the  CST  data,  treatment  effects  for  the  mathematics 
GAI  students  are  all  statistically  significant  and  about  two-thirds  of  a 
standard  deviation  above  the  adjusted  mean  of  students  in  other  CAI 
curriculums.     For  the  CTBS  data,  none  of  the  treatment  effects  is  statisti- 
cally significant,  although  the  treatment  effect  for  the  group  in  computation 
is  quite  strong* 

Table  4-21  presents  the  reading  results  for  fourth  graders  in  year 
4,     Statistically  significant  treatment  effects  appear  only  for  the  R 
group,  which  received  CAI  reading.    Although  the  treatment  effects  for  the 
L  (lauguage)  and  C  (reading  for  comprehension)  groups  are  both  strongly 
positive  when  compared  to  theM  group,  none  of  them  is  statistically 
significant.     For  the  CTBS  data,  all  of  the  treatment  effects  for  students 
receiving  reading,  language  or  reading-f or-comprehension  CAI  are  positive 
but  not  statistically  significant.    Table  4-22  shows  reading  results  for 
the  reading-f or-comprehension  CST,  which  was  developed  for  use  in  School 
1  only.    The  treatment  effect  for  group  C  is  statistically  significant, 
and  the  adjusted  mean  for  group  C  is  .60  of  a  standard  deviation  above 
the  adjusted  mean  for  group  M. 

Table  4-23  presents  the  language  results  for  grade  4,  year  4. 
For  the  CST  given  only  in  Schools  2-4,  treatment  effects  on  the  language 
total  are  statistically  significant  fox  both  the  language  group  and  the 
reading  group.    Adjusted  means  for  the  L  and  R  groups  are  .50  and  .80  of 
a  standard  deviation  above  that  of  the  group.    On  the  CTBS,  statistically 
significant  treatment  effects  exist  for  the  L  group  in  spelling,  mechanics 
and  language  total;  for  the  R  group  in  mechanics  and  language  total;  and 
for  the  C  group  in  language  expression.     All  the  rest  of  the  treatment 
effects  for  R  and  C  are  positive  but  not  significant. 

ER?C  27  J    '  : 
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Table  4-20 

One-Year  Analysis  for  GRADE  -  4  YEAR  -  4 

Posttest  -  CST  A!^D  CTBS  Subject  -  MATHEMATICS. 

Treatment  effects  and  post test  means  by  treatment  group, 
Schools  "  1-4. 


CST 

Treatment 
Group 

N 

Treatment 
Effect 

it ; 
294df 

Posttest 
X  SD 

Kesiuuai 
SD 

o tanaaraizea 

...  T.E. 

D  A  D  T  A 

M 

M 

/i 

40.23 

> 

11.88 

7  11 
/  •  1  J 

U  •  0  J 

DTP 

R,  Ly  L 

zuz 

v.. 

37.45 

11.06 

PART  B 

M 

1  1  o 

H.  IJ 

J.  U  / 

27.75 

12.60 

C    Q 1 

0.  ol 

n  AT 

R,L,C 

202 

25.87 

11.25 

MATH 

M 

117 

0.  oz 

J.  /  J 

67.97 

23.65 

IZ  • 

n  AQ 

U  •  07 

TOTAL 

R,L,C 

202 

63.31 

,21.25 

Treatment 

Treatment 

(t) 

Pdfettest 

Residual 

o  tanaaraizeo 

CTBS  . 

Group 

N 

Effect 

271df 

X 

SD 

SD 

T.E. 

COMPUTATION 

M 

i  UJ 

J  .  1  0 

1  • 

37.82 

9.20 

/.  JO 

U  .  ^4  J 

R 

U.  U  J 

36.00 

9.81 

u  •  Ul  ^ 

JO 

U.  00 

n  /i  ^ 

U.  H  J 

36.43 

10.63 

n  10 

C 

JO 

39.00 

8.52 

CONCEPTS 

M 

103 

0.27 

0.32 

17.12 

5.72 

4.00 

0.07 

R 

103 

-0.21 

-0.25 

17.16 

5.59 

-0.05 

L 

58 

-1.62 

-1.54 

16.02 

5.82 

-0.41 

C 

36 

20.03 

3.92 

APPLICATIONS 

M 

103 

0.20 

0.19 

13.77 

6.93 

4.91 

0.04 

R 

103 

-0.05 

-0.04 

14.08 

6.93 

-0.01 

L 

58 

-0.15 

-0.12 

13.74 

7.03 

-0.03 

C 

36 

18.14 

6.40 

MATH 

-  M 

103 

3.63 

1.25 

68.70 

19.46 

13.22 

0.27 

TOTAL 

R 

103 

-0.22 

-0.08 

67.25 

19.84. 

-0.02 

L 

58 

-0.89 

-0.26 

66.19 

21.07 

-0.07 

C 

36 

77.17 

17.13 

-62- 

Table  4-21 

One-Year  Analysis  for  GRADE  -  4  YEAR  -  4. 

Posttest  -  CST  AND  CTBS  Subject  -  READING. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  "  1-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
267df 


Posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


PART  A 


M 

L 
C 


101 
101 
62 
32 


2.97 
1.16 
2.08 


2.71 
0.88 
1.19 


47.81 
52.03 
47.45 
53.47 


11.38 
6.73 

11.79 
3.67 


7.58 


0.39 
0.15 
0;27 


PART  B 


M 
R 
L 
C 


101 

101 
62 
32 


3.49 
2.07 
2.90 


3.12 
1.55 
1.63 


31.83 
37.13 
32.31 
40.91 


12.76 
10.82 
13.59 
9.03 


7.71 


0.45 
0.27 
0.38 


READING 
TOTAL 


M 
R 
L 
C 


101 
101 
62 
32 


6.46 
3.23 
4.98 


3.46 
1.44 
1.67 


79.64 
89.16 
79.76 
94.374 


21.91 
15.96 
24.25 
11.91 


12.91 


0.50 
0.25 
0.39 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
260df 


_Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


VOCABULARY 


M 
R 
L 
C 


96 
101 
60 
36 


1.55 
2.11 
1.26 


1.77 
1.95 
0.95 


22.87 
23.77 
21.22 
29.14 


10.55 
9.58 
9.66 

■  7.77 


5.89 


0.26 
0.  36 
0.21 


COMPREHENSION 


M 
R 
L 
C 


96 
101 
60 
36 


1.34 
1.52 
2.14 


1.24 
1.14 
1.32 


24.87 
25.46 
23.05 
31.39 


11.51 
11.74 
12.69 
10.68 


7.25 


0.18 
0.21 
0.30 


READING 
TOTAL 


M 
R 
L 
C 


96 
101 
60 
36 


2.89 
3.63 
3.40 


1.74 
1.76 
1.35 


47.74 
49.24 
44.27 
60.53 


20.74 
20.32 
21.25 
17.23 


11.18 


0.26 
0.  32 

0.30" 


ERIC 
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Table  4-22 

One-Year  Analysis  for  GRADE  -  4  YEAR  -  4. 

Posttest  -  CST  Subject  -  READING. 

Treatment  effects  and- posttest  means  by  treatment  group, 
Schools  -  2-4. 


Subtest 


Treatment 
Group 


Treatment 
Effect 


(t) 
159df 


posttest 
X  SD 


Residual  Standardized 
SD  T.E.^  


PART  A 


M 
R 
L 


63 
59 

60 


3.77 
\.T. 


i.n 

1.07 


46.11 
51.15 
47.83 


12.88 
7.67 
11.21 


8.70 


0.43 
0.20 


PART  B 


M 
R 
L 


63 
59 
60 


3.83 
2.40 


2.35 
..1.51 


30.19 
35.42 
32.60 


12.99 
11.84 
13.03 


8.55 


0.45 
0.28 


TOTAL 


M 
R 
L 


63 
59 
60 


7.59 
4.14 


2.74 
1.52 


One-Year  Analysis  for  GRADE  -  4 


76.30 
86.58 
80.43 

YEAR  -  4. 


23.30 
17.96 
23.03 


14.57 


Posttest  -  CST 


Subject  -  READING  FOR  COMPREHENSION. 


Treatment  effects  and  posttest  means  by  treatment  group, 
School  -  1.  (School  -  1). 


0.52 
0.28 


Subtest 


Treatment 
Group 


Treatment 
Effect 


92J 


P^osttest 
X  SD 


Residual  Standardized 
SD  T.E. 


READING  FOR  M 
COMPREHENSION  R 
C 


33 
40 
34 


-0.48 
4.70 


■0.25 
2.33 


43.94 
44.28 
51.35 


14.32 
14.29 
11.92 


7.87 


-0.06 
0.60 


ERIC 
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Table  4-23 

One-Year  Analysis  for  GRADE  «  4  YEAR  »  4. 

Posttest  -  CST  AND  CTBS  Subject  -  LANGUAGE. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  ■  2-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
159df 


P^osttest . 
X  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


M 
R 
L 


63 
59 

60 


4.89 
5.45 


3.31 
3.80 


38.44 
44.42 
.43.77 


13.05 
8.79 
11.47 


7.71 


0.63 
0.71 


PART  B 


M 
R 
L 


63 
59 

60 


2.31 
6.14 


1.47 
3.99 


33.68 
37.52 
39.13 


11.88 
9.38 
12.95 


8.26 


0.28 
0.74  • 


LANGUAGE 
TOTAL 


M 
R 

L 


63 
59 

60 


7.17 
11.59 


2.61 
4.31 


72.13 
81.95 
82.90 


23.49 
16.97 
23.78 


14.41 


0.50 
0.80 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
260df 


£osttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


SPELLING 


M 
R 
L 
C 


96 
101 
60 
36 


1.54 
2.93 
1.00 


1.71 
2.62 
0.73 


29.06 
29.93 
29.07 
32.83 


9.78 
9.47 
8.85 
8.40 


6.06 


0.25 
0.48 
0.17 


MECHANICS 


M 
R 
L 
C 


96 

101 

60 
36- 


1.07 
1.44 
1.10 


2.18 
2.37 
1.48 


11.31 
12.20 
11.33 
14.00 


4.86 
4.63 
4.77 
4.31 


3.29 


0.33 
0.44 
0.33 


EXPRESSION 


M 
R 
L 
C 


96 

101 
60 
36 


1.08 
0.88 
1.99 


1.64 
1.08 
2.01 


29.06 
29.93 
29.07 
32.83 


9.78 
9.47 
8.85 
8.40 


4.40 


0.25 
0.20 
0.45 


LANGUAGE 
TOTAL 


M 
R 
L 
C 


96 
101 
60 
36 


3.68 
5.24 
4.08 


2.34 
2.69 
1.71 


58.85 
61.35 
57.17 
69.86 


20.09 
18.13 
18.40 
15.14 


10.58 


0.35 
0.50 
0 . 39 


ERIC 
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Summary  of  the  Grade  4  Data 

The  data  across  3  years  of  CAI  are  fairly  consistent,  with  the 
exception  of  the  mathematics  CTBS  data  in  year  1,  in  which  all  treatment 
effects  are  negative  for  MM  and  MRL  students.    The  mathematics  results 
in  grade  4,  year  1  are  puzzling..   Pretest  scores  on  the  ITBS  Level  9 
mathematics  subtests  were  the  highest  in  the  study.     Both  subtests 
contained  word  problems  which«required  reading  as  well  as  mathematical 
ability.     The  scores  of  bilingual  students  were  especially  high.  Since 
the  test  was  read  to  students  in  lower  grade  levels,  perhaps  in  some 
cases  the  mathematics  pretests  were  read  in  grade  4  in  year  1. 

Mathematics  results  in. years  3  and  4  are  more  consistent  with  data 
from  earlier  grades.    Mathematics  CAI  students  in  years  3  and  4  show 
consistently  positive  effects  with  statistically  significant  effects  on 
the  CSTs  and  the  computation  subtest  of  the  CTBS. 

The  reading  results  are  consistently  positive  except-  for  the  MRL 
group  in  year  1.     The  RL  students  in  years  1  and  3  and  the  reading  (R) 
students  received  roughly  equivalent  amounts  of  reading  CAI  before  being 
tested  on  the  reading  CST.     In  all  3  years  those  students  performed 
significantly  better  on  the  reading  CST  than  did  students  assigned  to 
mathematics  CAI.     They  also  performed  better  on  the  CTBS  reading  subtests. 
Although  only  one  of  the  treatment  effects  on  the  CTBS  is  statistically 
significant,  they  are  consistently  positive  with  test  scores  above 
1.24.  • 

The  language  results  are  even  stronger.    All  students  receiving 
language  CAI  and/or  reading  CAI  have  CAI  treatment  effects  on  the  language 
CST  which  are  positive  and  statistically  significant.    Except  for  the 


-66 

MRL  group  in  year  1,  all  CTBS  treatment  effects  are  positive,  and  treatment 
effects  on  the  CTBS  language  totals  are  statistically  significant. 

In  light  of  the  poor  performance  of  MM  students  in  year  1  after  only 
2  months  of  CAI,  the  performance  of  the  RL  group  in  year  1  is  surprising. 
The  reading, and  language  curriculums  demonstrated  their  treatment  effects 
much  more  quickly  than  the  MM  group.     Although  the  treatment  effects  on 
the  CTBS  after  only  two  months  of  CAI  failed  to  reach  statistical  signifi- 
cance, they  closely  parallel  the  results  of  students  in  years  3  and  4 
where  the  effects  are  often  statistically  significant. 


f 


The  Grade  5  Studies 

The  one-year  studies  for^  grade  5  and  curriculums  compared  are 
summarized  as  follows: 

Year  1  Year  2  Year  3  Year  A 

Cohort  CAI:    ,  Cohort  CAI: 

Grade  5         Controls  MM,  MRL,  RL         Controls  MM,  RL 

Comparisons         Comparisons  Comparisons  Comparisons 

The  Design  and  Analyses  for  Grade  5  ^ 

In  grade  5  all  three  CAI  curriculums  were  available — math,  reading, 
and  language.     In  year  2  the  design  for  the  fifth  grade  continued  the 
exposure  conditions  started  in  year  1  for  the  fourth  graders.    Thus  there 
are  three  conditions: 

MM  ■  two  10-minute  sessions  of  mathematics  daily, 

RL  ■  one  10-minute  session  of  reading  and  one  10-mirtftte 
session  of  language  daily, 

MRL  ■  one  10-minute  session  of  mathematics  followed  by 
one  10-minute  session  alternating  between  reading 
and  language. 

Fifth  graders  -who  had  been  randomly  assigned  to  one  of  these  three 

■  ■  S 

conditions  in  year  1  retained ^their  assignments,  and  new  fifth  graders 
were  randomly  assigned  to  one 'of  the  three  conditions. 

In  year  4,  the  design  for  the  fifth  graders  continued  the  exposure 
conditions  for  the  fourth  gradfers  in  year  3.     Students  were  randomly 
assigned  to  either  "MM,  two  10-minut^? session  of  mathematics  daily,  or  RL, 
one  10-minute  session  of  reading  and  one  10-minute  session  of  language 
daily.    The  MM  curriculum  concentrated  on  math  skills  while  the  RL 


curriculum  concentrated  on  verbal  skills*    This  provided  a  direct  compari- 
son of  the  effectiveness  of  each  of  these  two  types  of  curriculums. 

Results  for  Grade  5  - 

Year  2.     Table ^-24  gives  the  results  for  the  GST  and  GTBS  mathematics 
scores  for  fifth  graders  in  year  2\  ''For  the,  GST  .data,  the  treatment 
effects  for  the  MM  students  are  all  statistically  significant,  while  fdr  ^ 
the  MRL  students  only  one  part-score  fails  to  reach  significance* 
Treatment  effects  for  the  MM  condition  are  dovible  the  .treatment  effects 
of  the  MRL  condition*     In  terms  of  the  residual  standard  deviation,  MM 
students  are  *  78  of  a  standard  deviation  and  MRL  students  .SS  of  a 
standard  deviation  above  the  adjusted  mean  of  the  RL  group*"*  Of  the  GTBS 
subtests,  only  computation  shows  a^  statistically  significant  treatment 
effect  for  the  MM  group*    The  MRL  group  has  a  treatment  effect  in  computa- 
tion about  half  the  size  of  the  effect  for  the  MM  group,  but  it  fails  to 

r. 

reach  significance*  , 

Table  4-25  gives  the  results  for  the  GST  and  GTBS  reading  scores  for 
fifth  graders  in  year  2,    None  of.  the  treatment  effects  in  reading  are 
statistically  significant*  ^      ,  ' 

Table  4-26  presents  the  results  for  the  GST  and  PTBS  language 
scores  for  fifth  graders  in  year  2.     For  the  curriculum-specific  tests, 
the  MRL  condition  failed  to  produce  any  statistically  significant  treatment 
effects,  although  the^^  conditions  did*  ,  Students  who  received  the  RL 
treatment  performed  significantly  better  than  MM  students  on  the  language 
GSTs*    The  adjusted  mean  score  for  RL  students  is  * 74  of  a  standard 
deviation  above  the  adjusted  mean  for  the  MM  group*    On  the  GTBS  tests, 
however,  none  of  the  treatment  conditions  produced  significant  treatment 
effects  in  langua^  subtests* 
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Table  4-24 

One-Year  Analysis  for  GRADE  =  5  YEAR  »  2. 

Posttest-  CST  AND  CTBS  Subject  =  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  "1-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
163df 


2osttest 

X  SD 


Residual  Standardized 
SD  T.E. 


PART  A 


MM 

MRL 

RL 


61 
64 
56 


4.83  - 
2.10 


3.51 
1.62 


35.43 
28.91 
27.87 


12.05 
10.05 
10.05 


6.84 


0.71 
0.31 


PART  B 


MM 

MRL 

RL 


61 
64 
56 


3.93 
2.18 


3.50 
2.06 


31.93 
27.28 
25.78 


9.71 
10.04 
8.22 


5.58 


0.70 
0.39 


MATH 
TOTAL 


MM 

MRL 

RL 


61 
64 
56 


8.75 
4.28 


3.86 
2.00 


67.36 
56.19 
53.66 


21.15 
21.93 
17.44 


11.27 


0.78 
0.38 


CTBS 


Treatment 
Group 


COMPUTATION 


CONCEPT 


MM 

MRL 

RL 


MM 

MRL 

RL 


60 
67 
63 


60 
67 
63 


Treatment 
/  Effect 


2.33 
1.20 


-0.07 
-0.78 


(t) 
172df 


2.00 
1.07 


■0.13 
■l'.53 


P^osttest 
X  SD 


28.08 
25.73 
25.09 


12.73  , 

11.60 

12.68 


9.26 
9.34 
9.22 


4.17 
4.23 
4.30 


Residual  Standardized 
SD  T.E. 


6.24 


2.84 


0.37. 
0.19 


■0.02 
■0.27 


APPLICATION 


MM 

MRL 

RL 


60 
67 
63 


■0.32 
0.13 


-0.47 
0.20 


10.30 
10.13 
10.32 


5.52 
4.88 
5.19 


3.60 


■0.09 
0.04 


MATH 
TOTAL 


MM 

MRL 

RL 


60 
67 
63 


1.95 
0.55 


1.10 
0.32 


51.12 
47.46 
48.09 


16.56 
16.56 
16.26 


9.47 


0.21 
0.06 


FRir 


2^1 
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/  Tatle  4-25 

/ 

One-Year  Analysis  for  GRADE  -  5  YEAR  =  2. 

Posttest-  CST  AND  CTBS  Subject  -  READING. 

Treatment  effects  aild  posttest  means  by  treatment  group. 
Schools  =1-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
150df 


posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


PART  A 


MM 

MRL 

RL 


55 
63 
52 


-0.32 
1.29 


-0.23 
•0.92 


36.87 
35.67 
41.11 


10.48 
11.34 
9.86 


6.85 


-0.05 
0.19 


PART  B 


MM 

MRL 

RL 


55 
63 
52 


-0.89 
1.48 


-0.60 
0.99 


32.03 
30.09 
37.27 


12.16 
13.41 
11.08 


7.29 


-0.12 
0.20 


READING 
TOTAL 


MM 

MRL 

RL 


55 
63 
52 


-1.21 
2.77 


-0.47 
1.07 


68.91 
65.76 
78.38 


21.99 
23.65 
20.10 


12.69 


-0.10 
0.22 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
I49df 


posttest 

X     ^  SD 


Residual  Standardized 
SD  T.E. 


VOCABULARY 


COMPREHENSION 


MM 

MRL 

RL 


MM 

MRL 

RL 


54 
61 
56 


54 
61 
56 


0.13 
-0.16 


-1.31 
-0.49 


0.14 
-0.16 


-1.27 
-0.45 


20.31 
19.74 
21.69 


21.83 
20.128 
22.178 


7.32 
8.95 
8.53 


8.54 
9.99 
9.00 


4.65 


5.26 


0.03 
-0.03 


-0.25 
-0.09 


READING 
TOTAL 


MM 

MRL 

RL 


54 
61 
56 


-1.17 
-0.64 


-0.71 
-0.37 


42.15 
40.16 
44.48 


14.85 
18.30 
16.72 


8.48 


-0.14 
-0.08 


ERIC 
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Table  4-26 

One-Year  Analysis  for  GRADE  -  5  YEAR  -  2. 

Posttest-  CST  Al^D  CTBS  Subject  -  LANGUAGE. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -  1-4. 


CST 


Treatment 
GroUp 


Treatment 
Effect 


(t) 
150d.f 


posttest 
y  SD 


Residual 
SD 


S  tandardized 
T.E. 


PART  A 


PART  B 


LANGUAGE 
TOTAL 


55 
63 
52 


55 
63 
52 


55 

63 
52 


2.14 
4.38 


-0.48 
3.36 


1.66 
7.75 


1.66 
3.39 


-0.38 
2.64 


0.77 
3.59 


39.20 
41.32 
46.36 


37.12 
37.01 
43.33 


76.33 
78.33 
89.69 


10.55 
9.35 
7.83 


8.22 
9.87 
8.41 


17.42 
17.98 
15.20 


6.31 


6.22 


10.52 


0.34 
0.69 


-0.08 
0.54 


0.16 
0.74- 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
149df 


JPosttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


SPELLING 


MM 

MRL 

RL 


54 
61 
56 


■0.79 
1.09 


■0.80 
1.05 


32.80 
32.28 
36.27 


7.31 
9.21 
8.54 


5.05 


-0.16 
0.22 


MECHANICS 


MM 

MRL 

RL 


54 
61 
56 


0.48 
0.47 


0.84 
0.78 


10.70 
11.05 
11.84 


3.95 
4.86 
3.93 


2.92 


0.16 
0.16 


EXPRESSION 


MM 

MRL 

RL 


54 
61 
56 


-1.06 
-1.59 


■1.30 
■i.86 


17.74 
16.34 
17.36 


6.63 
6.91 
6.98 


4.17 


-0.25 
-0.38 


LANGUAGE 
TOTAL 


MM 

MRL 

RL 


54 
61 
56 


-1.37 
-0.04 


-0.80 
-0.02 


61.24 
59.-67 
65.46 


15.55 
18.96 
17.17 


8.73 


-0.16 
0.00 


ERIC 


2rr) 
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Year_4.     Table  4-27  presents  the  results  for  CST  and  CTBS 
mathematics  tests  for  fifth  graders  in  year  4,     For  the  CSTs,  the  treat- 
ment effects  for  the  MM  group  are  all  statistically  significant.  The 
adjusted  mean  for  the  CST  total  for  the  MM  group  is  .76  of  a  standard 
deviation  above  the  adjusted  mean  for  the  RL  group.     I'or  the  CTBS  data, 
only  the  treatment  effect  for  computation  is  statistically  significant 
and  favors  the  MM  group  by  half  a  standard  deviation. 

Table  4-28  presents  the  results  for  CST  and  CTBS  reading  scores 
for  fifth  graders  in  year  4,    Again,  there  are  no  statistically  significant 
treatment  effects  although  all  are  positive.     The  reading  total  for  the  CTBS 
approaches  significance,  with  the  adjusted  mean  score  for  the  RL  group  about 
one-third  of  a  standard  deviation  higher  than  the  adjusted  mean  for  the  MM  group. 

Table  4-29  presents  the  results  for  CST  and  CTBS  language  tests  for 
fifth  graders  in  year  4.     For  the  CSTs,  the  treatment  effects  for  the  RL 
group  are  all  statistically  significant  with  the  performance  of  the  RL  group 
about  one-half  a  standard  deviation  above  the  performance  of  the  MM  group. 
Although  the  treatment  effects  on  the  CSTs  are  positive  overall  for 
the  RL  group,  none  of  them  achieves  statistical  significance. 

Summary  of  the  Grade  5  Data 

The  results  for  grade  5  in  years*"2  and  4  are  remarkably  similar.     In  both 
years  most  students  were  repeating  earlier  CAI  assignments  while  new  students 
were  randomly  assigned.    Although  there  were  three  treatment  conditions  in 
year  2~MM,  MRL  and  RL— only  two  of  these  were  used  in  year  4:    MM  and  RL.  In 
both  years  the  MM  group  performed  significantly  better  on  the  CSTs  and  the 
CTBS  computation  subtest.     In  both  years  the  only  statistically  significant 
treatment  effects  for  the  RL  group  are  on  the  language  CSTs.    Although  many 
reading  and  language  treatment  effects  are  positive,  none  but  the  language  CST 
achieves  statistical  significance. 


ERIC  "^'U 
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Table  4-27 

One-Year  Analysis  for  GRADE  -  5  YEAR  ■  4. 

Posttest-  CST  AND  CTBS  Subject  -  MATHEMATICS. 

Treatment  effects  and  pbsttest  means  by  treatment  group. 
Schools  "  2-4. 


CST 


Treatment 
Group 


Treatment  (t) 
Effect  146df 


P^osttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


MM 
RL 


101 
63 


4.22 


3.42 


36.62 
30.35 


11.35 
11.93 


7.01 


0.60 


PART  B 


MM 
RL 


101 
63 


5.09 


4.38 


34.34 
27.52 


10.21 
9.48 


6.60 


0.77 


MATH 
TOTAL 


MM 
RL 


101 
63 


9.31 


4.31 


70.96 
59.87 


20.73 
20.57 


12.27 


0.76 


CTBS 


Treatment  Treatment  (t) 

Group        .     _N_         Effect  142df 


posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


COMPUTATION 


MM 
RL 


94 
67 


3.61 


2.99 


29.39 
24.91 


8.63 
9.01 


6.98 


0.52 


CONCEPT 


MM 
RL 


94 
67 


-0.32  -0.56 


12.15 
12.04 


4.70 
4.70 


3.35 


-0.10 


APPLICATION 


MM 
RL 


94 
67 


-0.58  -0.91 


11.03 
11.06 


5.65 
5.39 


3.66 


-0.16 


MATH 
TOTAL 


MM 
RL 


94 

67  - 


2.70 


1.44 


52.57' 
48.01 


16.51 
16.78 


10.89 


0.25 


2^  ^ 


ERJC 
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Table  4p28 

One-Year  Analysis  for  GRADE  -  5  YEAR  -  4. 

Posttest-  CST  AND  CTBS  -    I  Sub ject  -  READING. 

i 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  ■2-4. 


CST 


PART  A 


Treatment 
Group 


MM 
RL 


Treatment  (t) 
N  Effect  128df 


87 
61 


0.98 


0.85 


JPosttest 
X  SD 


38.93 
40.06 


10.38 
'9.57 


Residual  Standardized 
SD  T.E. 


6.22 


0.16 


PART  B 


MM 
RL 


87 
61 


1.36 


1.07 


35.86 
37.13 


11.80 
11.51 


6.90 


0.20 


READING 
TOTAL 


CTBS 


MM 
RL 


Treatment 
Group 


87 
61 


2.35 


i 


Treatment 
Effect 


1.06 


(t) 
138df 


74.79 
77.20 


21.40 
20.59 


posttest 
X  SD 


11.97 


Residual 
SD 


0.20 

Standardized 
T.E. 


VOCABULARY 


MM 
RL 


94 
67 


1.05 


1.35 


18.02 
18.73 


8.34 
8.04 


4.50 


'0.23 


COMPREHENSION 


MM 
RL 


94 
67 


1.48 


1.81 


18.19 
19.04 


8.56 
8.74 


4.74 


0.31 


READING 
TOTAL 


MM 
RL 


94 
67 


2.52 


1.91 


36.21 
37.78 


15.99 
15.88 


7.67 


0.33 
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Table  4-29 

One-Year  Analysis  for  GRADE  -  5  YEAR  =  4. 

Posttest-  CST  AND  CTBS  Subject  -  LANGUAGE. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  «  2-4. 


CST 


Treatment  Treatment  (t) 

Group  N  Effect  128df 


posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


PART  A 


MM 
RL 


87 
61 


1.91 


1.99 


41.18 
43.77 


7.20 
8.31 


5.20 


0.37 


PART  B 


MM 
RL 


87 
61 


2.85 


2.95 


37.76 
40.54 


8.34 
8.16 


5.22 


0.55 


LANGUAGE 
TOTAL 


MM 
RL 


87 
61 


4.76 


3.03 


78.94 
84.31 


14.29 
15.31 


8.49 


0.56 


CTBS 


Treatment  Treatment  (t) 

Group  N  Effect  138df 


posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


SPELLING 


MM 
RL 


94 
67 


-0.54  -0.55 


32.00  7.65  .5.73 
30.24  8.22 


-0.09 


MECHANICS 


MM 
RL 


94 

67 


0.66 


1.24 


10.87 
11.01 


4.29 
4.18 


3.09 


0.21 


EXPRESSION 


MM 
RL 


94 
67 


1.22  1.93 


16.33 
16.82 


5.99 
6.48 


3.66 


0.33 


LANGUAGE 
TOTAL 


MM 
RL 


94 
67 


1.33  0.87 


59.20 
58.07 


15.79 
16.13 


8.90 


0.15 


ERIC 
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The  Grade  6  Studies 
The  one-7ear  studies  for  grade  6  are  summarized  as  follows: 

Year  1  Year  2   Year  3  .  Year  A 

CAI:  Cohort  CAl:  ^  Cohort 

Grade  6         MM,  RR,  RL,  LL        Controls  MM,  MRL,  RL  Controls 

Comparisons  Comparisons       Comparisons  Comparisons 

Three  curriculums— Tnathematics,  reading  and  language — were  all 
available  for  grade  6.     Different  combinations  of  these  three  curriculums 
were  studied  to  determine  the  effectiveness  of  CAI. 

The  Design  and  Analyses  for  Grade  6 

Year  !•     In  the  first  year,  the  assumption  was  made  that  reading  and 
language  CAI  curriculums  covered  different  skills.    All  sixth  graders 
were  given  two  10-minute  sessions  of  CAI  daily,  but  there  were  four 
treatment  conditions*    These  conditions  were  as  follows: 


MM  Mathematics^Mathematics 

RR  Reading-Reading 

RL  Reading^Language 

LL  Language-Language 


The  four  treatments  at  the  sixth-grade  level  allowed  a  separate 
comparison  of  the  accomplishments  of  students  in  the  reading  and  language 
CAI  curriculums*     Students  were  randomly  assigned  to  one  of  the  four 
treatment  curriculums  in  CAI  schools*     Because  the  CAI  treatment  started 
in  February  of  year  1  and  CTBS  tests  were  given  in  April,  each  student 
received  8  to  10  weeks  of  CAI  before  CTBS  testing*     The  CSTs  were 
given  in  late  May  after  three  months  or  more  of  CAI* 
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The  research  design  at  the  sixth  grade  in  year  1  was  compromised  by 
the  actual  conditions  during  implementation.    Although  students  were 
randomly  assigned  to  each  of  the  four  conditions,  brighter  students 
topped  out  of  the  language  and  reading  curriculums.     Some  students  topped 
out  during  the  rapid-motion  phase  in  the  first  10  sessions  of  CAI;  some 
topped  out  much  later.    Although  students  who  topped  out  and  were  reassigned 
to  mathematics  were  screened  out  of  the  analyses,  the  randomized  design 
was  compromised.     Students  remaining  in  the  RR  and,  especially,  the  LL 
conditions  were  the  slower  students.     Students  in  the  RL  condition  did 
not  top  out  as  frequently  because  they  spent  only  half  as  much  time  in 
each  of  the  curriculums.     Therefore  the  RL  students  are  likely  to  be  a 
brighter  group  than  the  RR  or  LL  groups.     One  hopes  the  regression 
analyses  compensate  for  the  randomization  problems,  but  we  recognize  the 
deficiencies  of  the  design  at  this  level. 

Year  3.    Most  students  in  grade  6  in  CAI  schools  in  year  3  already 
had  been  exposed  to  one-'and-one-half  years  of  computer-assisted  instruction. 
In  grade  4  the  students  had  been  assigned  to  one  of  three  treatment 
conditions:     two  sessions  of  math  CAI  daily  (MM),  one  session  of  reading 
and  one  of  language  daily  (RL),  or  a  combination  of  one  session  of  math 
daily  with  a  second  session  which  alternated  between  reading  and  language 
(MRL).     In  grade  5  and  again  in  grade  6,  returning  students  were  given 
their  original  CAI  treatments  while  new  students  were  randomly  assigned 
to  one  of  the  three  conditions.     Because  of  the  LAUSD  desegregation  plan, 
there  were  no  sixth  graders  in  School  1  in  year  3. 


Results  for  Grade  6 

Year  1.    Table  A-30  presents  the  results^  for  CST  and  CT.BS  mathematics 
scores.    For  the  CST  data,  the  treatment  effects  for  the  MM  group  are 
statistically  significant.    For  the  mathematics  CST  data,  the  adjusted 
mean  for  the  MM  group  is  almost  one"and"one  half  standard  deviations 
above  the  adjusted  mean  for  the  other  groups  combined.    There  are  no 
statistically  significant  differences  between  the  MM  group  and  other 
groups  on  the  CTBS  mathematics  subtests.     Since  the  CTBS  was  administered 
only  a  few  weeks  after  the  onset  of  CAI,  the  failure  to  find  difference;? 
is  not  surprising. 

Table  4-31  gives  the  results  for  the  CST  and  CTBS  reading  tests. 
For  the  CST  data,  statistically  significant  differences  in  the  treatment 
effects  are  found  for  the  RR  group  on  part  B  and  the  total  and  for  the  RL 
group  on  part  B.     Overall  the  RR  group  did  about  twice  as  well  as  the  RL 
group  when  compared  to  the  MM  group  on  the  reading  CST.     The  CTBS  reading 
results  may  reflect  some  treatment  effects  of  exposure  to  the  CAI  curricu" 
lum  as  well  as  the  topping  out  phenomenon  described  earlier.    Only  the 
treatment  effect  for  the  RL  group  in  reading  comprehension  is  statisti- 
cally significant,  although  all  are  positive. 

Table  4-32  presents  the  results  for  the  CST  and  CTBS  language 
tests.    For  the  CST  data,  treatment  effects  for  the  LL  and  the  RL  groups 
are  all  statistically  significant,  but  there  appeares  to  be  little 
difference  between  them.    As  we  have  seen  earlier,  students  receiving 
half  as  much  CAI  perform  about  half  as  well.     This  may  indicate  a 
ceiling  effect  for  the  test  and  the  CAI  curriculum.     None  of  the  treatment 
effects  for  the  CTBS  language  subtests  is  statistically  significant. 

Overall,  results  for  the  CSTs  in  year  1  are  significant  and . in  the 
predicted  direction.     The  CTBS  results  are  not  significant,  as  might  have 
been  expected  since  exposure  to  CAI  was  limited. 
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Table  4-30 

One-Year  Analysis  for  GRADE  -  6  YEAR  -  1. 

Posttest  -  CST  AND  CTBS  Subject  -  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  "  1-4. 


CST 


Treatment 
Group 


Treatment  (t) 
N  Effect  176df 


posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


PART  A 


MM 

RR.LL.RL 


60 
134 


6.87  8.46 


35.35 
25.47 


10.66^ 
5.94 


4.89 


1.40 


PART  B 


MM 

RR.LL.RL 


60 
134 


'5.28 


7.06 


27.92 
18.62 


10.57 
5.61 


5.35 


1.17 


MATH 
TOTAL 


MM 

RR.LL.RL 


60 
134 


13.15 


8.65 


63.27 
44.08 


20.79 
10.19 


9.15 


1.44 


CTBS 


Treatment 
Group 


Treatment  (t) 
N  Effect  183df 


posttest 
X  SD 


Residual  Standardized 
Sp   T.E. 


COMPUTATION  MM  63 

RR.LL.RL  138 


1.04 


0.96 


34.43 
30.36 


8.36 
7.77 


6.56 


0.16 


CONCEPTS 


MM 

RR.LL.RL 


63 
138 


-0.27  -0.55 


14.41 
13.05 


4.60 
3.78 


2.91 


-0.09 


APPLICATION          MM  63 
RR.LL.RL  138 


-0.39  -0.56 


14.41 
12.34 


5.97 
5.25 


4.13 


-0.09 


MATH 
TOTAL 


MM 

RR.LL.RL 


63 
138 


0.39 


0.22 


63.25 
55.75 


17.16 
14.17 


10.80 


0.04 
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Table  4-31 


One-Year  Analysis  for  GRADE  •  6  YEAR  -  1. 

Posttest  •  CST  AND  CTBS  Subject  •  READING. 

Treatment  effects  and  posttest  means  by  treatment  group. 
Schools  *  1-4. 


Treatment 

Treatment 

(t) 

Postt 

est 

CST 

Group 

N 

Effect 

192df 

X 

SD 

PART  A 

MM 

56 

35. 38 

'  10.26 

RR 

49 

2. 67 

1.83 

36.82 

10.64 

RL 

61 

1.15 

0.84 

35.84 

10.53 

LL 

49 

-2.36 

-1.61 

34.20 

11.25 

PART  B 

MM 

56 

25.54 

11.06 

RR 

49 

4.37 

3.32 

28.67 

10.92 

RL 

61 

2.49 

1.99 

27.49 

11.44 

LL 

49 

0.72 

0.54 

27.94 

12.40 

READING 

MM 

56 

60.91 

19.91 

TOTAL 

RR 

49 

7.04  ' 

2.82 

65.49 

20.79 

RL 

61 

3.64 

1.54 

63.33 

20.94 

LL 

49 

-1.64 

-0.65 

62.14 

22.58 

* 

Treatment 

Treatment 

(t) 

Postt 

est ' 

CTBS 

Group 

N 

Effect 

183df 

X 

SD 

VOCABULARY 

MM 

65 

23.18 

8.69 

RR 

53 

1.17 

1.38 

21.42 

7.24 

RL 

51 

1.24 

1.44 

21.61 

7.14 

LL 

34 

-0.71 

-0.73 

19.74 

8.20 

COMPREHENSION      MM  65               -  23.66  10.26 

RR  53  2.56  2.36  23.25  7.78 

RL  51  2.86  2.61  23.57  6.74 

LL  34  0.49  0.39  21.44  8.79 


READING  MM  65  -  46.85  18.25 

TOTAL  RR  53  3.73         2.24  44.66  13.80 

RL  -   51.  4.10         2.43  45.18  13.16 

LL  34  -0V23  0.12  -  41.18  16.32 


Residual  Standardized 
SD  T.E. 


0.37 
0.16 
-0.33 


0.67 
0.38 
0.11 


0.57 
0.29 
-0.13 


Residual  Standardized 
SD  T.E. 


4.44 

0 . 26 
0.28 
-0.16 


5.64 

0.45 
0.51 
0.09 


8.67 

0.43 

0.47^ 
-0.03 


6.55 


12.45 
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Table  4-32 

One-Year  Analysis  for  GRADE  ■=  6  YEAR  -  !• 

Posttest  -  CST  AND  GTBS  Subject  «  LANGUAGE. 


Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  "  1-4. 


CST 

Treatment 
Group 

JL 

Treatment 
Effect 

(t) 
192df 

Posttest 
X  SD 

Residual 
SD 

Standardized 
T.E. 

PART  A 

MM 

56 

39.30 

6.67 

3.  lo 

RR 

49 

0.46 

0.44 

38.80 

7.66 

0.09 

RL 

61 

4.21 

4.27 

43.00 

7.38 

0.81 

T  L 

49 

4.52 

4.32 

44.37 

.  6.59 

0.87 

PART  B 

MM 

56 

34.04 

8.05 

5.27 

RR 

49 

0.60 

0.57 

33.78 

6.68 

0.11 

RL 

61 

3.65 

3.64 

37.20 

7.61 

0.69 

LL 

49 

3.90 

3.67 

38.45 

8.58 

q .  74 

LANGUAGE 

MM 

56 

73.34 

13.64 

8.98 

TOTAL 

RR 

49 

1.06 

0.59 

72.57 

13.38 

0.12 

RL 

61 

7.86 

4.59 

80.20 

14.20 

0.88  ' 

LL 

49 

8.42 

.  4.64 

82.82 

14.10 

0.94 

Treatment* 

Treatment 

(t) 

Posttest 

Residual 

Standardized 

CTBS 

Group 

N 

Effect 

173df 

X 

SD 

SD 

T.E. 

SPELLING 

MM 

64  ^ 

35.47  ' 

8.77 

0.  u  / 

rr' 

54 

-0.48 

-0.41 

32.44 

8.03 

-u.uo 

RL 

46 

0.  0^ 

33.35 

6.37 

LL 

31 

-0.62 

-0.44 

31.35 

7.79  . 

-0.10 

MECHANICS 

MM 

64 

10.53 

4.34 

2.95 

-  RR 

.  54 

0.85 

1.50  . 

10.98 

3.64 

0.29 

RL 

46 

0.53 

0.89 

10.57 

.  3.40 

0.18 

LL 

31 

0.25 

0.36 

9.58 

4.04 

0.08 

EXPRESSION 

MM  d 

64 

19.13 

7.11 

4.57 

RR  ^ 

54 

0.58 

0.66 

18.15 

5.16 

0.13 

RL 

46 

0.53 

0.58 

17.93 

4.78 

0.12 

LL 

31 

-0.40 

-0.37 

16.45 

6.44 

-0.09 

LANGUAGE 

m 

64 

65.13 

17.71 

10.^18 

TOTAL 

RR 

54 

0.96 

0.49 

61.57 

14.43 

0.09 

RL 

46 

1.76 

0.85 

61.85 

10.66 

0.17 

LL 

31 

-0.78 

-0.32 

57.39 

14.61 

-0.08 

ERIC 
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statlstlcally  slgnlfi- 
the  MM  group. 


Year  3,    Table  4-33  presents  the  results  for  CST  and  CTBS  mathematics 
scores  for  sixth  graders  in  year  3.    Most  student;s  were  in  their  third 
year  of  CAI  with  the  same  treatment  conditions.     For  the  CSTs  the  tr^tment 
effects  for  the  MM  and  MRL  groups  are  statistically  significant  overall 
as  compared  to  RL  students.     In  general,  treatment  effects  for  the  MRL 
group  are  about  half  those  of  the  MM  group.     For  the  CTBS  mathematics 
data  there  are  no  statistically  significant  treatment  effects,  although 
the  strongest  effect  is  for  tlje  MM  group  in  computations 

Table  4-34  gives  the  results  for  the  CST  and  CTBS  reading  scofes. 
For  the  CST  data,  none  of  the  treatment  effects  is 
cant,  although  differences  favor  the  RL  group  over 
For  the  CTBS  data,  treatment  effects  for  the  RL  and  MRL  groups  in  reading 
vocabulary  are  statistically  significant,  with  thei  adjusted  means  for  the 
RL  and  MRL  groups  almost  half  a  standard  deviation  above  the  adjusted 
means  for  the  MM  group. 

Table  4-35  presents  the  results  for  the  CST  and  CTBS  language 
scores.     For  the  CST  data,  treatment  effects  for  the  RL  group  are  statis- 
tically significant  for  part  A  and  the  reading  total.     In  general, 
treatment  effects  for  the  MRL  group  are  about  half  of  the  treatment 
effects  for  the  RL  group.    The  adjusted  mean  for  the  RL  group  is  .53  of 
a  standard  deviation  higher  than  the  adjusted  mean  for  the  MM  group. 
None  of. the  treatment  effects  for  CTBS  scores  is  statistically  significant. 


ERIC 
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lable 4-33 

One-Year  Analysis  for  GRADE  -  6  YEAR  -  3. 

Posttkst-  CST  AND  CTBS  Subject  -  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -  2-4. 


CST 


PART  A 


Treatment 
i  Group 


MM 

MRL 

RL 


55 
54 
41 


Treatment 
Effect 


4.74 
3.20 


(t) 
133df 


3.59 
2.54 


P^osttest 
X  SD 


,37.69 
35.39 
32.71 


11.07 
10.17 
9.19 


Residual 
SD 


5.97 


Standardjized 
T.E. 


0.7? 
0.|5A 


PART  B 


MM 

MRL 

RL 


55 
54 

41 


3.89 
1.80 


3.23 
1.56 


27.62 
25.54 
24.12 


11.25 
10.02 
7.63 


5.45 


0.71 
0.33 


MATH 
TOTAL 


CTBS 


MM 

MRL 

RL 

Treatment 
Group 


55 
-54 
41 


8.62 
5.01 


Treatment 
Effect 


3.75 
2.27 


(t) 
133df 


65.30 
60.92 
56.83 


21.83 
19.66 
16.05 


10.43 


P^osttest 
X  SD 


0.83 
0.48 


Residual  Standardized 
SD  T.E. 


COMPUTATION 


MM 

MRL 

RL 


51 
55 
44 


2.39 
1.17 


1.51 
0.77 


33.21 
31.16 
30.68 


9.94 
8.61 
8.57 


7.25 


0.  33 
0.16 


CONCEPTS 


MM 

MRL 

RL 


51 
55 
44 


0.16 
■0.29 


0.22 
-0'.42 


14.39 
13.58 
14.32 


5.23 
4.64 
4.08 


3.36 


0.05 
-0.09 


APPLICATIONS 


MM 

MRL 

RL 


51 
55 
44 


1.16 
0.94 


1.20 
1.02 


13.65 
12.65 
12.16 


6.20 
6.47 
5.50 


4.44 


0.  26.. 
0.21 


MATH 
TOTAL 


MM 

MRL 

RL 


51 
55 
44 


3.72 
1.82 


1.37 
0.70 


61.25 
57.40 
57.16 


19.58 
18.03 
15.37 


12.41 


0.30 
0.15 


ERIC 
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Table  4-34 

One-Year  Analysis  for  GRADE  -  6  YEAR  -  3. 

I      .  .  ■ 

Posttest-  CST  AND  CTBS  Subject  -  READING. 

•  Tre^atment  effects  and  posttest  means  by  treatment  group, 
Schools  ■ '2-4. 


CST 


Treatment 
Group 


Treatment  (t) 
Effect  122df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


MM 

MRL 

RL 


45 
53 
43 


1.49 
2.23 


1.16 
1.60 


37.71 
38.91 
41.40 


11.28 
9.97 
8.35 


5.87 


0.25 
0.38 


PART  B 


MM 

MRL 

RL 


45 
53 
43 


0.47 
2.56 


0.33 
1.65 


31.18 
31.36 
34.56 


11.17 
12.59 
10.67 


6. 54 


0.07 
0.39 


READING 
TOTAL 


MM 

MRL 

RL 


45 
53 
43 


1.96 
4.79 


0.82 
1.83 


68.89 
70.26 
75.95 


21.43 
21.82 
17.68 


10.97 


0.18 
9.44 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
128df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
f'.E. 


VOCABULARY 


MM 

MRL 

RL 


50 
56 
43 


2.58 
2.28 


2.44 
2.03 


21.48 
23.21 
24.33 


9.29 
9.26 
8.99 


5.06 


0.51 
0.45 


COMPREHENSION 


MM 

MRL 

RL 


50 
56 
43 


-0.36 
0.18 


-0.30 
0.14 


24.54 
23.36 
25.28 


11.06 
10.98 
9.66 


5.71 


■0.p6 
0.03 


READING 
TOTAL  " 


MM 
.  MRL 
"  RL 


50 
56 
43 


2.22 
2.46 


1.10 
1.15 


46.02 
46.57 
49.60 


19.69 
19.61 
17.92 


9.62 


0.23' 
0.26 


ERIC 
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Table  4-35' 

;  ■  i  ■    .  , 

One-Year  Analysis  for  GRADE  •  6  YEAR  «=  3.  | 

Posttest  »  CST  AND  CTBS  Subject  -  LANGUi^GE. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -  2-4. 


CST 


Treatment  Treatment  (t) 

Group  N  Effect  122df 


posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


MM  i 
MRL 
RL  i 


45 

53 
43 


0.98 
2.75 


0.98 
2.52 


41 . 00 
.41.96 
44.95 


8.45 
7.56 
6.37 


4.58 


0.21 
0.60 


PART  B 


MM 

MRL 

RL 


45 
53 

43 


0.93 
1.34 


1.02 
1.35 


36.18 
37.02 
38.88 


7.52 
6.93 
5.42 


4.17 


0.22 
0.32 


LANGUAGE 
TOTAL 


MM 

MRL 

RL 


45 
53 
43 


1.91 
4.08 


1.14 
2.24 


77.18 
78.98 
83.84 


15.23 
13.80 
11.25 


7.64 


0.25 
0.53 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
128df 


posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


SPELLING 


MM 

MRL 

RL 


50 
56 
43 


-0.84 
-0.19 


-0.64 
-0.14 


33.92 
34.09 
35.58 


8.98 
9.95 
9.39 


6.27 


-0.13 
-0.03 


MECHANICS 


MM 

MRL 

RL 


50 
56 
43 


0.21 
.0.65 


0.32 
0.93 


11.64 
11.75 
12.60 


4.36 
4.62 
3.69 


3.14 


0.07 
0.21 


EXPRESSION 


MM 

MRL 

RL 


50 
56 
43 


0.75 
0.86 


0.82 
0.88 


18.76 
19.23 
20.35 


7.97 
7.91 
7.60 


4.36 


0.17 
0.2O 


LANGUAGE 
TOTAL 


MM 

MRL 

RL 


50 
56 
43 


0.12 
1.32 


0.05 
0.54 


64.32 
65.07 
68.53 


18.81 
20.90 
18.59 


10.93 


0.01 
0.12 
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Summary  of  the  Grade  6  Data 

For  the  sixth-grade  data,  the  CSTs  show  that  students  assigned  to 
a  CAI  curriculum  did.  In  fact,  perform  better  when  tested  on  that  curriculum* 
Results  are  statistically  significant  for  the  mathematics  and  language 
CSTs  and  for  some  of  the  reading  CSTs. 

For  the  CTBS  data  only  one  year's  results  were  applicable.     In  year  3, 
RL  students  performed  significantly  better  on  reading  vocabulary 
than  did  the  MM  students.    The  MM  students  performed  better  on 'mathematics 
computation,  although  the  effect  is  not  statistically  significant. 
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Chapter  IV 
LONGITUDINAL  EXPERIMENTAL  STUDIES 

Introduction 

One  of  the  goals  of  this  CAI  study  is  to  assess  the  effects  of  more 
than  one  year  of  exposure  to  the  three  CAI  curriculums.     This  requires  a 
longitudinal  design  in  which  CAI  is  used  with  the  same  group  of  students  . 
as  they  proceed  through  successive  grades.     We  have  already  seen  all  of 
the  components  of  the  longitudinal  studies  as  we  reviewed  the  12  one-year 
studies.     A  schematic  description  of  the  longitudinal  design  is  as  follows; 


Grade  1    Grade  2    Grade  3    Grade  4    Grade  5    Grade  6 

4 


Year  1 


Year  2 


Year  3 


Year  4 


The  letters  indicate  the  treatments  received  by  the  students  at  designated 
years  and  grades.     Solid  lines  indicate  that  individual  students  were 
assigned  to  the  same  treatment  groups  in  the  subsequent  years.     Dashed  lines 
indicate  that  treatment  groups  were  re-randomized  in  the  succeeding  year. 

r 

There  are  three  longitudinal  studies:    grade  1  to  grade  3,  grade  2 /.to  grade 
5,  and  grade  4  to  grade  6.     Besides  allowing  us  to  study  the  effect  of 
long-term  exposure  to  various  CAI  curriculums,  this  longitudinal  design 
provides  the  opportunity  of  assessing  the  effects  of  changing  curriculum 
jnixes  on  some  of  the  students  in  different  years.    The  design  also  gives  ; 


us  some  replication  of  certain  aspects  of  the  study  under  somewhat  different 
conditions. 

<i 

The  Grade  1  to  Grade  3  Longitudinal  Study 

The  grade  1  to  grade  3  longitudinal  study  is  summarized  as  follows: 
Grade  1  '       Grade  2         Grade"  3 

where  0  stands  for  zero  CAI  curriculum  exposure.     For  first  and  second 
grades »  mathematics  was  the  only  curriculum  available.     Zero  CAI  exposure 
(0)  ..was  used  as  a  control  for  mathematics  CAI  exposure.     For  third 
graders^  mathematics  and  language  curriculums  were  compared.     Unlike  the 
other  two  longitudinal  *studies»  the  procedures  to  randomize  the  assignment 
of  curriculums  to  students  were  activated  in  each  of  the  three  years. 
For  the  fbur  CAI  schools  in  year  2,  classrooms  at  the  first-grade 

level  were  randomly  selected  to  receive  or  not ^receive  10  minutes  of 

f* 

mathematics  CAI  curriculum.    Pairs  of  classrooms  were  identified  on  the 
basis  of  a  priori  information.     One  number  of  each  pair  of  classrooms  was. 
selected  for  CAI;  the  other  was  its  control.     In  year  3»  classrooms  at 
the  second-^rade  level  were^again  randomly  divided  into  two  categories; 
one  received  the  CAI  mathematics  curriculum  the  other  did  not.  Within 
the  CAI  classrooms,  students  in  the  mathematics  CAI  curriculum  were 
randomly  assigned  to  either  fixed  or  variable  strands.  The  overall 
curriculum  .was  the  same  but  the  method  of  presentation' was  different. 
Variable  strands  presented  questions  from  all  available  strands  of  the 
mathematics  curriculum  in  variable  order »  while  fixed  strands  allowed  the 
teacher  to  select  the  strand  in  which  the  student  worked. 


In  year  4,  both  mathematics  and  language  arts  curriculums  were  available 
for  third  graders,  and  they  were  randomly  assigned  within  classrooms  to 

either  mathematics  CAIor  language  arts  CAI  for  10  minutes-  daily.     The  third- 

j 

grade  design  provided  a  completely  randomized  comparison  between  the i 
mathematics  and  language  curriculums. 

.  Because  the  randomization  procedutes  occurred  every  year,  thercj  were 
many  combinations  of  CAI  treatment  sequences  across  the  three-year  | 
experimental  period.     For  example,  therfe  were  four  treatment  sequences 
across  grades  1  and  2  if  we  do  not  distinguish  between  fixed  and  variable 
strands.'    There  were  00,  OM,  MO  and  MM,  where  0  stands  for  no  CAI  ^xposure. 
This  two-year  longitudinal  study  provided  a  comparison  of  the  benefits  of 
first  and/or  second  grade  CAI  instruction 'in  mathematics.     The  language 
CAI  curriculum  for  third  graders  in  year  4  served  as  a  control  for  the 
mathematics  CAI  treatment  effect.    By  the  end  of  grade  3  there  were  eight 
possible  CAI  sequences  for  students  in  the  grades  1-3  CAI  cohort. 


1  -  *  ■ 

i* 
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Results  for  the  Longitudinal  Data>  Grades  1-3 

The  analysis  of  the  data  for  this  longitudinal  group  Is  complicated 
by  the  fact  that,  at  the  end  of  the  third  year  of  the  study,  there  were 
eight  treatment  conditions  among  less  than  100  CAI  students  who  remained. 

Four  sets  of  analyses  will  be  described  on  the  following  pages: 
jgrades  1-2  analyses,  grades  1-3  analyses,  CAI  students  vs.  cohort  controls, 
I  arid  CAI  students  vs.  comparisons. 

Table  4-36  presents  the  results  for  the  CST  and  CTBS  mathematics 
scores  of  students  In  CAI  schools  In  grades  1  and  2.     By  the  end  of  grade 
2  there  were  four  treatment  groupp:     some  students  had  no  mathematics  CAI 
either  year  (bO),  some  had  CAI  only  In  grade  1  (MO),  some  had  CAI  only  In 
grade  2  (OM),  and  some  had  CAI  both  years  (MM).    The  CST  results  show 
that  treatment  effects  for  the  OM  and  MM  groups  are  statistically 
significant  when  compared  to  the  group  with  no  CAI.     In  terms  of  the 
residual  standard  deviation,  the  adjusted  means  of  groups  MO,  OM,  and  MM 
are  .25,  .71,  and  .66  of  a  standard  deviation  above  the  adjusted  mean  for 
students  without  CAI. 

Results  for  the  CTBS  show  that  the  treatment  effects  for  the  MM 
group  for  computation  and  mathematics  total  are  both  statistically 
significant.     The  treatment  effects  are  orderly;  I.e.  ,  the  more  mathematics 
CAI  or  the  more  recent  the  CAI  experience,  the  higher  the  treatment 
effect.    For  the  CTBS  mathematics  total,  the  adjusted  means  for  the  MO, 
OM,  and  MM  groups  are  .25,   .53^  and  .71  of  a  standard  deviation  above 
the  adjusted  mean  for  the  students  without  CAI. 
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Table  4-36 

Longitudinal  Analysis  for  GRADE  1  to  GRADE  2. 
Period  =  YEAR  2  to  YEAR  3. 

Posttest  «  CST  AND  CTBS«  Subject  -=  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -1-4. 


Treatment  . 
CST  Group 

Treatment 
EfXrecu 

(t) 

Posttest 
X  SD 

Residual 

Standardized 
■  T.E. 

PART  A 

00 

27 

33.48 

11.43 

9.  06 

MO 

22 

2.60 

0.95 

34.59 

10.52 

0. 

29 

OM 

37 

4.67 

1.81 

39.70 

11.07 

0. 

51 

MM 

25 

5.32 

2. 02 

37.84 

9.05 

0. 

59'. 

PART  B 

-  00 

27 

19.26 

12.98 

9.54 

MO 

22 

1  f)8 

0.  58 

20.05 

10.07 

0. 

18 

OM 

37 

7.66 

2.82  •■ 

30.19 

13.30 

0. 

80 

MM 

9  S 

6  1  7 

2  '22 

25.04 

13.05 

0. 

65 

MATH 

00 

27 

52I74 

23.15 

17.40 

TOTAL 

MO 

A  97 

n  ft! 

54.64 

19.90 

0. 

25 

(M 

37 

12.32 

2.49 

69.89 

23.15 

0. 

71 

MM 

9  S 

1 1  iif) 

2  27 

62.88 

21 .42 

0. 

66 

Treatment 

Treatment 

(t) 

Posttest 

Residual 

Standardized 

CTBS 

Group 

N 

Effect 

104df. 

X 

SD 

SD 

T. 

E. 

COMPUTATION 

GO 

31 

14.39 

6.90 

5.17 

MO 

21 

1.83 

1.18 

15.05 

6.70 

0. 

35 

OM 

40 

2.30 

1.60 

18.18' 

6.59 

0. 

44 

MM 

25 

3.47 

2.38 

18.00 

6.18 

0. 

67 

CONCEPTS  AND 

GO 

31 

■  13.94 

5.54 

'3.68 

APPLICATIONS 

MO 

21 

-0.13 

-0.11 

13.24 

5. 56 

-0. 

03 

OM 

40  ■ 

1.64 

1.61  . 

16.85 

5.30 

0. 

45 

MM 

25 

1.85 

1.78 

15.32 

4.06 

0. 

50 

MATH 

00 

31 

28.32 

11.29 

7.50 

TOTAL 

MO 

21 

1.70 

0.75 

28.29 

10.66 

0. 

23 

m 

40 

3.94 

1.90 

35.03 

11.26 

0. 

52 

MM 

25 

5.31  ^ 

2.51 

33.32 

9.31 

0. 

71 

i 

3  jt  • 

) 
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Table  4-37  presents  the  results  for  the  CST  and  CTBS  mathematics 
scores  for  longitudinal  students  in  grades  1-3.    Whereas-the  two-year 
regressions  showed  four  groups  of  students,  the  number  is  doubled  for  the 
three-year  regressions.     Students  in  each  of  the  previous  four  groups 
were  assigned  either  mathematics  or  language  XAI  in  grade  3.  Table  4-37 
collapses  the  eight  groups -of  students  into  four  groups  as  follows:*' 

OOL  *?  0  years  of  mathematics  CAI 
MOO,  OMO,.  OOM  -  1  year, of  mathematics  CAI 
MOM,  OM^,  MMO'-  2  years  "  " 
MMM  ■  3  years 

The  CST  results  show  statistically"  signif icant  treatment  effects  for 
most  of  the  CAI  conditions,,  and  the  treatment  effects  are  ordered.  ^  The  - 
more  mathematics  CAI,  the  greater  the  treatment  effect.     For  the  total 
CST,  the  adjusted  means  for  students  with  one,   two  or  three  years  of 
mathematics  CAI  are  .74,  .91,  and  1.54  of  a  standard  deviation  above  the  ^ 
adjusted  me^n  for  students  without  CAI. 

The  CTBS  results  are  more  confused.    Many  bilingual  students  we're 
not  tested  with  the  CTBS  in  the  final  year  of  the  study.     The  decision 
not  to  test  bilingual  students  in  one  school  was  beyond' the  cojitrol  of 
the  CAI  project.     It  was  unfortunate  in  that  the  loss  of  students  reduced 
the  number  of  students  with  three  years  of  CAI  to  four.     None  of  the 
treatment  effects  for  CTBS  subtests  is  significant.     Nevertheless  it  is 
interesting  to  note  that  for  computation,  the  adjusted  means  for  students 
with  one  and  two  years  of  CAI  are  .29  and  .61  of  a  standard  deviation 
above  the  adjusted  mean  for  students  with  no  CAI  experience.    Those  results 
are  not  statistically  significant,  but  they  do  follow  the  general  trend. 
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Table  4-37 

Longitudinal  Analysis  for  GRADE  1  to  GRADE  3. 
Period  -  YEAR  2  to  YEAR  4. 

Post  test  »  CST  Al^D  CTBS  Subject  -  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  ■  1-4 • 


Treatment 

Treatment 

(t) 

Posttest 

Residual 

Standardized 

Group 

Vi 

Eiiiect 

X 

SD 

SD 

T.E. 

PART  A 

0 

14 



29.57 

14.04 

1 

27 

8.61 

2.43 

45.11 

11.02 

,  9.21 

0.93 

2 

32 

8.89 

2.73 

40.50  -, 

14.41 

0.96 

3 

12 

12.20 

2.71 

42.17 

10:42 

1.32  ' 

PART  B 

0 

14 

20.14 

9.30 

1 

27 

3.79 

1.07 

28.44 

13.92 

9.21 

0.  U\ 

2 

32 

6.35 

1.95 

28.44 

14.81 

0  69 

3 

12 

13.44 

2.98 

30.58 

12.50 

'  1.46 

MATH 

0 

14 

49.71 

22.68 

TOTAL 

1 

27 

12.40 

1.93 

7,9.30 

24.33 

16.67 

0  lU 

2 

32 

15.25 

2.59 

68.94 

27.70 

0  91 

3 

12 

25.64 

3.15 

72.75 

20.36 

1.54 

Treatment 

Treatment 

Ct) 

Posttest 

Residual 

Standardized 

CTBS 

Group 

N 

Effect 

46df 

X 

SD 

SD 

T.E. 

• 

COMPUTATION 

0 

13 

- 

21.85 

10.02 

«• 

1 

24 

1.81 

0.67 

33.96 

10.51 

6.27 

0.29 

2 

•  26 

3.81 

1.49 

27.15 

11.82 

0.61 

3 

\4 

\ 

0.89 

0.21 

25.50 

11.41 

0.14 

CONCEPTS 

0 

13 

9.69 

6.18 

1 

24 

0.10 

0.07 

16.42 

,6.44 

3.31 

0.03 

2 

26 

1.45 

1.08 

12.62 

6.62 

0.44 

3 

4 

0.12 

0.05 

11.50 

5.85 

0.04 

APPLICATIONS 

0 

13 

7.77 

5.42 

1  ' 

24 

0.48 

0.31 

14.75 

5.68- 

3.57 

0.13 

2 

26 

0.66 

0.46 

10.54 

6.78 

0.18 

'  3 

4 

-0.85 

-0.36 

8.00 

7.71 

-0.24 

MATH 

0 

13 

39.31 

20.47  " 

TOTAL 

1 

24 

2.40 

0.51 

65.13 

21 . 06 

10.84 

0.22 

2 

26 

5.93 

1.34 

■  50.31 

24.26 

0.55 

3 

4 

0.15 

0.02 

■  45.00 

23.33 

0.01 

ERIC 
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Table  A-38  gives  the  results  for  CST  and  CTBS  mathematics  scores 
for  students  in  the  grades  1-3  longitudinal  CAI  group  compared  to  students 
in  the  cohort  control  group  in  the  same  schools  a  year  earlier.    We  have 
just  looked  at  results  for  the  CAI  group,  and  we  know  that  some  students 
in  that  group  had  no  CAI  at  all  while  others  had  from  one  to  three  years 
of  CAI.     In  these  analyses,  we  make  no  differentiation  among  CAI  students. 
We  simply  compare  the  total  group  with  those  who  were  in  grades  1-3  in  " 
the  same  schools  one  year  earlier. 

The  CST  data  show  that  the  treatment  effects  for  the  CAI  group  are 
statistically  significant.    When  the  treatment  effects  are  expressed  in 
terms  of  the  residual  standard  deviation,  the  CAI  group  is  .80  of  a 
standard  deviation  higher  than  their  cohort  controls. 

The  CTBS  data  likewise  show  that  the  treatment  effects  for  the  CAI 
group  are  statistically  significant.    Adjusted  means  for  the  CAI  group 
are  from  .38  to  .51  of  a  standard  deviation  above  the  adjusted  means  for 
the  cohort  controls. 


T 
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Jable  4-38 

Longitudinal  Malysis  for  GRADE  1  to  GRADE  3:  CAI  VS  COHORT. 

Period  -  YEAR  1  to  YEAR  3  FOR  COHORT.    YEAR  2  to  YEAR  4  FOR  CAI. 

Posttest  »  CST  AND  CTBS  Subject  -  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  •'1-4. 


CST 


Treatment 
Group 


N 


Treatment 
Effect 


(t) 
210df 


Posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


PART  A 


CAI 
COHORT 


97 
125 


8.23 


5.34 


40.96 
38.19 


13.85 
13.77 


10.61 


0.  78 


PART  B 


CAI  97 
COHORT  .  125' 


7.00 


4.79 


29.75 
26.76 


13.80 
11.46 


10.08 


0.69 


MATH 
TOTAL 


CAI 
COHORT 


97 
125 


15.23 


5.55 


70.71 
64.95 


26.38 
23.74 


18.93 


0.80 


CTBS 


Treatment 
Group 


N 


Treatment 
Effect 


(t) 
185df 


Posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


COMPUTATION 


CAI 
COHORT 


79 

118 


4.46 


3.28 


28.91 
28.45 


11.35 
12.03 


8.72 


0.51 


CONCEPTS 


CAI 
COHORT 


79 
118 


1.99 


2.95 


.14.18 
14.27 


6.38 
6.15 


4.31 


0.46  - 


APPLICATION 


CAI 
COHORT 


79 
118 


1.87 


2.46 


11.53 
11.63 


6.82 
6.81 


4.88 


0.38 


MATH 
TOTAL 


CAI 
COHORT 


79 
118 


8.33 


3.44 


54.62 
54.35 


23'.  12 
23.06 


15. .48 


0.54 
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Table  4-39  gives  the  results  for  CST  and  CTBS  mathematics  scores 

0 

for  students  in  the  grades  1-3  longitudinal  CAI  group  compared  to  students 
in  two  comparison  schools* 

For  the  CST  data,  the  treatment  effects  for  the  CAI  group  are  all 
statistically  significant.  The  CAI  group  overall  is  .76  of  a  standard 
deviation  above  the  adjusted  mean  of  the  comparison  group* 

For  the  CTBS.  data,  significant  treatment  effects  are  found  for 
concepts,  applications,  and  the  CTBS  total.    For  the  total  score,  the  CAI 
group  is  about  half  a  standard  deviation  higher  than  the  comparison 
. group. 
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Table  4-39 

Longitudinal  Analysis  for  GRADE  1  to  GRADE  3:    CAI  VS  COMPARISON. 

Period  -  YEAR  1  to  YEAR  3  FOR  COMPARISON.    YEAR  2  to  YEAR  4  FOR  CAI. 

Posttest  -  CST  AND^CTBS  Subject  -  MATHEMATICS." 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -  1-4  VS  5-6.  ,  -  ■■ 


CST 


Treatment 
Group 


N 


Treatment 
Effect 


(t) 
199df 


Posttest 
X     -  SO 


Residual 
SO 


Standardized 
T.E. 


PART  A 


CAI  97 
COMPARISON  111 


9.28 


5.45 


40.96 
34.18 


13.85 
12.37 


11.21 


0.83 


PART  B 


CAI  97 
COMPARISON  111 


6.09         3.88         29.75        13.80  -10.32 
_       •  25o51  10.79 


0.59 


MATH 
TOTAL 


CAI  97 
COMPARISON  111 


15.37         4.98  70.71        26.38  20.31 

59.69  22.42 


0.76 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
178df 


posttest 
X  SO 


Residual  Standardized 
SO  T.E. '  ' 


COMPUTATION    .      CAI  79 
COMPARISON  108 


1.37 


0.88 


28.91 
29.50 


11.35 
10»74 


9.29 


0.15 


CONCEPTS 


CAI  79 
COMPARISON  108 


2.91  3.55 


14.18 
12.29' 


6.38 
5.35 


4.85 


0.60 


APPLICATIONS        CAI  19 
COMPARISON  108 


3.91  5.15 


11.53 
8.31 


6.83 
'4.74 


^  4..50_ 


0.87 


MATH 
TOTAL 


CAI  ?9 
COMPARISON  108 


8.20 


3.09 


54.62 
50.09 


23.12 
17.64 


15.71 


0.52 
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Discussion  for  Grades  1*"3  -  .  * 

The  treatment  effects  for  students  assigned  to  mathematics  CAI  are 
remarkably  consistjant.    The  CST  results  for  CAI  students  across  three 
years  average  .SO  of  a  standard  deviation  above  the  adjusted  mean  for 
students  with  no  CAI.-   When  CAI  students  are  compared  with  cohort  controls 
in  the  same  schools  a  year  earlier  or  with  students  in  comparison  schools, 
CAI  students  again  are  .SO  or ^,76  of  a  standard  deviatidn  higher  on  the 
CST»     Clearly,  CAI  students  are  acquiring  the  skills  on  which  they 
receive  drill  and  practice  in  the  CAI  mathematics  curriculum. 

k    The  CTBS  results  are  less  remarkable  but  nevertheless  give  evidence 
that  drill  and  practice  CAI  helps  students!  performance  oil  standardized 
tests*    The  adjusted  mean  score  for  CAI  students  averages  about  half  a 
standard  deviation  above  the  adjusted  mean  score  for  cohort  controls  or 
comparison-school  students  on  the  mathematics  CTBS  total.' 


ERIC 
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The  Grade  2  to  Grade  5  Longitudinal  Study 

The  griide  2  to  grade  5  longitudinal  study  is  summarized  as  follows: 
tirade  2         Grade  3         Grade  4         Grade  5 


The  purpose 


M  ■^...^^^^>  MM. 
MM..^>^  ^  ML 


MM 
RL 


MM 
RL 


of  this  longitudinal  study  was  to  compare  the  pure  mathematics 


curriculum  versus  the  reading  and  language  combined  curriculums.  The 
longitudinal  study  started  in  year  1.     Second  graders  were  randomly 
assigned  to  one  or  two  sessions  of  mathematics         during  the  second 
semester.    As  has  been  mentioned  before,  this  design  was  extremely 
'^disruptive  to  second-grade  classrooms  and  met  with  a  good  deal  of  teacher' 
opposition.     In  grade  3,  students  were  assigned  either  two  sessions  of 
mathematics  CAI  or  one  session  of  mathematics  and  one  of  language  CAI. 
Several  options  had  been  contemplated,  including  a  different  design  in 
school  3,  until  January  1978,  when' trea.tments  were  finalized  as  described. 
Students  who  had  received  mathematics  and  language  CAI  in  grade  3 
received  reading  and  language  CAI  in  grade  4.     In  year  4,  the  fifth 
graders  continued  jthe  same  treatment  as  they  received  in  fourth  grade. 
Although  student-s  had  been  exposed  to  CAI  on  a  daily  basis  in  grades  2 
and  3,  in  grade  4 • classrooms  were  assigned  to  the  CAI  lab  either  on 
Monday,  Wednesday  and  Friday,  or  Tuesday  and  Thursday.     In  grade  5,  the, 
schedule  was  back  to  a  daily  basis.  "The  grade  3  to  5  segment  of  the  • 
longitudinal  study  was  selected  for  major  emphasis  because  of  the  initial 


startup  problems 


Ln  grade  2  and  because  from  grades  3-5?  the  two  CAI 


groups  began  to  increase  their  differentiation.     The  MM  group  retained 

the  double  dose  of  mathematics  treatment  for  three  consecutive  years 

while  the  other  longitudinal  group  switched  to  the  reading  and  language 
curriculums. 
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Results  for  the  Longitudinal  Data,  Grades  2-5 

Five  sets  of  analyses  will  be  presented  in  this  section.    First  we 
will  pres^ent  the  regression  for  CAI  students  from  grades  3-5,  comparing 
those  who  received  only  mathematics  CAI  with  those  who  received  mathematics/ 
language  in  grade  3  and  reading/ language  in  grades  4  and  5.  Because 
attrition  had  reduced  the  number  of  students  for  the  three^^ear  regression 
to' a  maximum  of  91,  two  two-7ear  regressions  are  reported  for  graded  3-4 
(N"l43)  and  grades  4-5  (N-106).     Finally  we  will  present  two  sets  of 
regressions  comparing  CAI  students  with  cohort  controls  in  the  same 
schools  a  'year  earlier  and  with  students  in  comparison  schools. 

Grades  "3-5  CAI.    Table  4-40  presents  the  results  for  the  CST  and 
CTBS  mathematics  scores  for  longitudinal  CAI  students  in  grades  3-5.  Fot 
the  CST  data,  treatment  effects  for  both  parts  and  the  total  CST  are 
statistically  significant.    The  MM  students  are  more  than  one  standard 
deviation  above  the  RL  students  on  the  CST  even.  thiDugh  the  RL  students 
had,  in  fact,  been  exposed  to  the  mathematics  CAI  curriculum  for  a  year 
in  grade  3.     For  the  CTBS  data,  the  treatment  effect  for  the/ computation 
subtest  is  statistically  significant,  with  MM  students  about  three-quarters 
of  a  standard  deviation  above  the  RL  group.    The  rest  of  the  treatment 
effects  for  the  CTBS  subtests  and  to):al  are  not  statistically  significant. 

Table  4-41  presents  the  results  for  the  CST  and  CTBS  reading  tests. 
None  of  the  treatment  effects  is  statistically  significant. 

Table  4-42  presents  the  results  for  the  CST  and  CTBS  language  tests. 
For  the  .CST. data,  the  treatment  effects  for  the  RL  group  in  language  are 
all  statistically  significant,  with  the  performance  of  the  RL  group  almost 
three-quarters  of  a  standard'  deviation  above  that  of  the  MM  group.  Ttaatment 
effects  for  the  CTBS  subtests  are  all  positive  but  none  is  statistically 
significant.  - 
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'Table  4t40 

Longitudinal  Analysis  for  GRADE  3  to  GRADE  5. 
PERIOD  -  YEAR  2  to  YEAR  4. 

Posttest-  CST  AND  CTBS  Subject  -  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  "  1-A.         "  ' 


CST 


Treatment 
Group 


Treatment 
^  Effect 


(t)^ 
52df 


posttest" 
X        '  SD 


Residual  Standardized 
SD-  T.E.  • 


PART  A 


MM 
RL 


57 
30 


8.05  -  4.23 


38.40 
32.53 


10.55 
10.37 


6.81 


1.18 


PART  B 


MM 
RL 


57 
30 


6.05       .  3.29 


35.44 
29.87 


10.26 
8.66 


6.57 


0.92 


MATH 
TOTAL 


MM 
RL 


57 
30 


14.10  4.17 


73.84 
62.40 


19.78 
18.07 


12.09 


1.17 


CTBS 


Treatment ' 
C^roup 


Treatment 
Effect 


(t) 
54df 


posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


COMPUTATION  MM 
RL 


58 
32 


5.46  2.89 


29.95 
25.41 


8.56 
9.^0. 


6.88 


0.79 


CONCEPTS 


MM 
RL 


58 
32 


-0.37    .  -0.37 


12.02 
12.53 


5.04 
S.09 


3.64 


-0.10 


APPLICATIONS 


MM 
RL 


58 
32 


-0'.23  -0.20 


11.26 
11.38 


5.68 
5.79 


4.26 


-0.05. 


MATH 
TOTAL 


MM 
RL 


58 
32 


4.86  1.48 


53.22 
49.31 


16.54 
17.69 


11.93 


0.41 
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Table  4-41 

Longitudinal  Analysis  for  GRADE  3  to  GRADE  5. 
PERIOD  -  YEAR  2  to  YEAR  4. 

Posttest-  CST  AND  CTBS  Subject  -  READING. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  "1-4. 


CST 


Treatment  Treatment  (t) 

Group  ■  _N__        .Effect  51df 


Posttest 
X  SD 


Residual  Standardized 


SD 


T.E. 


PART  A 


MM 
RL 


55 
30 


2.20 


1.19 


39.40 
42.40 


10.08 
7.23 


6.88 


0.32 


PARt  B 


MM 
RL 


55 
30 


2.03 


0.93 


36.06 
39.43 


11.85 
9.55 


8.10 


0.25 


READING 
TOTAL 


MM 
RL 


55 
30 


4.23 


1.17 


75.46 
81.83 


20.89 
15.84: 


13.49 


0.31 


CTBS 


Treatment 
Group 


Treatment 
Effect 


,(t) 
53df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


VOCABULARY 


MM 
RL 


58 
33 


-0.17  -0.14 


18.41 
19.46 


8.60 
8.39 


4.69 


-0.04 


COMPREHENSION 


MM 
RL 


58 
33 


1.17 


0.87 


18.59 
19.00 


9.44 
9.38 


4.94 


0.24 


READING 
TOTAL 


MM 
RL 


58 
33 


0*99 


0.44 


37.00 
38.46 


17.20 
16.92 


8.30 


0.12 
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Ta^le  4-42 

Longitudinal  Analysis  for, GRADE  3  to  GRADE  5. 
PERIOD  -  YEAR  2  to  .YEAR  4.-  , 
Posttest-  CSTAND  CTBS.     '  Subject  -  LANGUAGE. 


Schools  - 


CST  - 


PART  A 


PART  B 


/  LANGUAGE 
TOTAL 


CTBS 


SPELLING 


Treatment 

Group  ^  'N 

/ 

..MM 
RL 


MM 
RL 


MM 

RL, 


MM 
RLi 


MECHANICS       "  M^ 


EXPRESSION 


55 
30 


Treatment 

Group  /  N 


LANGUAGE 
TOTAL 


58 
33 


58 
33 


58 
33 


58 
33 


Effect 

(t) 
51df 

Posttest 
X  SD 

Residual 
•  SD 

Stajl 

1 

/  - 

i 

idardized 
T.E. 

2.17 

41.33 
45.63 

7.58 
8.16 

'^.29 

1 

0.58 

2.32 

38.22 
42.33 

8.03 
7.34 

5.33 

0.62 

D.  yo 

2. 66 

79.55 
87.96 

14.18 
14.22 

9.80 

0.71 

Treatment 
Effect 

(t)  - 
53df 

Posttest 

X       -   SD  - 

Residual 
SD 

Standardized 
/  T.E. 

0.33 

0.20 

32.02 
32.82 

8.27 
8.12 

5.95 

/ 

0.06 

1.28 

11.33 
11.76 

4.26 
3.78 

3.23 

0.40 

0.95 

0.89 

16.32 
17.00 

6.25 
6.81 

4. 00 

0.24 

2.56 

.  1.00 

59.66 
61.58 

.  16.65 
15.49 

9.46 

■  0.27 

o  / 
ERIC  , 
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Grades  3'"4  CAI>    On  the  next  few  pages  are  the  data  for  a  two-year 
longitudinal  study  of  grades  3  to  4  in  years  2  to  3.    Table  4-43  presents 
the  results  for  the  CST  and  CTBS  mathematics  dependent  variables •  For 
the  CST  data,  all  of  the  treatment  effects  for  the  MM  group  are  statisti- 
cally significant, with  the  performance  of  the  MM  group  about  one  standard 
deviation  higher  than  the  performance  of 'the  RL  group.    For  the  CTBS 
data,  treatment  effects  for^ computation  and  for  ^he  mathematics  total 
are  both  statistically  significant,  with  the  MM  group  more  than  half  a 
standard  deviation  above  the  RL  group.    The  treatment  effects  for  the 
remaining  CTBS  subtests  are  positive  but  not  statistically  significant. 

Table  4-44  gives  the  results  for  the  CST  and  CTBS  reading  dependent 
variables.    None  of  the  treatment  effects  is  statistically  significant. 

Table-  4-45  gives  the.  results  for  the  CST  and  CTBS  language  dependent 
variables.     For  the  CST  data,  all  of  the  treatment  effects  for  the  RL 
group  are  statistically  significant.    The  adjusted  mean  of  the  RL  gr^up 
is  more  than  half  a  standard  deviation  above  that  of  the  M^  group.  For 
the  CTBS  data,  none  of  the  treatment  effects  is  statistically  significant, 
although  the  treatment  effect  for  mechanics  approaches  significance. 
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1  Table  4-43 

Longitudinal  Analysis  fior  GRADE  3  to  GRADE  4, 
PERIOD  -  YEAR  2  to  YEAR  3. 

Posttest-  CST  AND  CTBSi  Subject  -  MATHEMATICS, 

Treatment  effects  and  posttest  means  by  treatment  group> 
Schools  -  1-4, 


CST 


Treatment 
Group 


N 


Treatment 
Effect 


.(t) 
102df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


MM 
RL 


/87 

'47 


7.34 


5.12 


42.55 
33.85 


10.25 
11.04 


7.21 


1.02 


PART  B 


MM 
RL 


87 

"47 


4.83 


3.76 


28.64 
22.58 


11.92 
11.12 


6.46 


0.75 


MATH 
TOTAL 


MM 
RL 


87 
47 


12.16 


4.97 


71.19 
56.43 


21.31 
21.52 


12.32 


0.99 


Treatment 
Subtest  Group 


Treatment 
Effect 


Ct) 
105df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


COMPUTATION  MM 
RL 


90 
50 


4.60 


3.08 


38.70 
35.02 


9.30 
10.54 


7.77 


0.59 


CONCEPTS 


MM 
RL 


90 
50 


0.90 


1.07 


16.67 
16.44 


6.06 
6.33 


4.33 


0.21 


APPLICATIONS 


MM. 
RL 


90 

50 


1.62 


1.88 


14.73 
14.18 


6.76 
7.58 


4.48 


0.36 


MATH 
TOTAL 


MM 
RL 


90 
50 


7.12 


2.76 


70.10 
65.64 


19.61 
22.53 


13.39 


0.53 


ERIC 
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Table  4-44 

Longitudinal  Analysis  for  GRADE  3  to  GRADE  4. 
PERIOD  -  YEAR  2  to  YEAR  3. 


Posttest-  CST  AND  CTBS 


Subject  -  READING. 


Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  "1-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
104df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


MM 
RL 


89 
48 


1.04 


0.64 


48.51 
50.65 


10.89 
7.87 


8.25 


0.13 


PART  B 


RL 


89 
48 


1.26 


0.82 


33.80 
35.83 


12.63 
10.00 


7.80 


0.16 


READING 
TOTAL 


MM 
RL 


89 
48 


2.30 


0.82 


82.30 
86.48 


22.11 
16.41 


14.29 


0.16 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
107df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


VOCABULARY 


MM 
RL 


89. 
53 


0.19 


0.20 


22.44 
22.28 


10.04 
11.04 


5.05 


0.04 


COMPREHENSION 


MM 
RL 


89 
53 


0.10 


0.08 


25.90 
25.81 


12.04 
12.49 


6.41 


0.02 


READING 
TOTAL 


MM 
RL 


89 

53 


0.29 


0.15 


48.34 
48.09 


21.42 
22.19 


10.04 


0.03 


ERIC 
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Table'  4-45 

Longitudinal  Analysis  for  GRADE  3  to  GRADE  4. 
PERIOD"  -  YEAR  2  to  YEAR  3. 

Posttest-  CST  AND  CTBS  Subject  -  LANGUAGE. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -  1-4. 


CST 


Treatment 
Group 


Treatment  (t) 
Effect  104df 


posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


PART  A 


MM 
RL 


89 
48 


2.98 


2.05 


42.43 
46.71 


10.28 
7.51 


7.38 


0.40 


PART  B 


MM 
RL 


89 
AS 


4.90 


3.59 


36.20 
42.17 


10.42 
9.18 


6.92 


0.71 


LANGUAGE 
TOTAL 


MM 
RL 


89 
48 


7.89 


3.00 


79.25 
88.88 


19.83 
16.15 


13.31 


0.59 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
107df 


Posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


SPELLING 


MM 
RL 


89 
53 


-1.61  -1.19 


29.11 
27.06 


10.11 
10.30. 


7.19 


-0.22 


MECHANICS 


MM 
RL 


89 
53 


1.11 


1.80 


11.18 
12.06 


4.84 
4.79 


3.26 


0.34 


EXPRESSION 


MM 
RL 


89 
S3 


0.72 


0.80 


17^67 
18.04 


7.69 
7.34 


4.74 


0.15 


LANGUAGE 
TOTAL 


MM 
RL 


89 

53 


0.21 


0.10 


57.97 
57.15 


20.64 
20.15 


11.63 


0.02 


32o 

o 

ERIC 
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Grades  4-5  CAI.    On  the  next  few  pages  are  the  data  for  a  two-year 
longitudinal  study  of  grades  4  to  5  in  years  3  to  4,    Table  4-46  presents 
the  results  for  the  CST  and  CTBS  tnathematics  performance.     For  the  CST 
data,  all  of  the  treatment'  effects  for  the  MM  group  are  statistically 
significant,  and  the  performance  of  the  MM  s^tudents  is  about  one  standard 
deviation  higher  than  the  performance  of  RL  students.     For  the  CTBS  data, 
only  the  treatment  for  computation  is  statistically  significant,  with  the 
performance  of  MM  students  more  than  half  a  standard  deviation  above  that 

» 

of  the  RL  students. 

Table  4-47  presents  the  results  for  the  CST  and  CTBS  reading  dependent 
variables.     For  the  CST  data,  all  of  the  treatment  effects  are  statistically 
.significant,  with  the  performance  of  the  RL  group  about  three-quarters  of 
a  standard  deviation  above  that  of  the  MM  group.     None  of  the  treatment 
effects  for  the  RL  group  on  CTBS  reading  subtests  is  statistically 
significant. 

Table  4-48  gives  the  results  for  the  CST  and  CTBS  language  subtests. 
For  the  CST,  all  of  the  treatment  effects  are  statistically  significant. 
The  adjusted  mean  for  the  RL  group  on  the  total  CST  is  more  than  one 
standard  deviation  above  the  adjusted  mean  for  the  MM  group.     None  of  the 
treatment  effects  for  the  RL  group  on  the  CTBS  language  subtests  is 
significant.  n 
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Table  4-46 

Longitudinal  Analysis  for  GRADE  4  to.  GRADE  5.  • 

PERIOD  -  YEAR  3  to  YEAR  4.  " 

Posttest-  CST  Al^D  CTBS  Subject  -  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group. 
Schools  "1-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
67df 


Posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


PART  A 


MM 
RL 


64 
32 


6.71 


4.59 


38.41: 
32.28 


11.04 
9.82 


5.74 


1.17 


PART  B 


MM 
RL 


64 
32 


5.16  3.50 


35.78-  10.36 
30.09  8.64. 


5.79 


0  .  89 


MATH 
TOTAL 


MM 
RL 


64 

32 


11.87  4.66 


:  74.19        20.47  10.02 
62.38  17,51 


1.18 


CTBS 


Treatment 
Group 


Treatment  (t) 
Effect    -  66df 


Posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


COMPUTATION 


MM 
RL 


64 

30 


3.81 


2.24 


29.11 
25.27 


8.65 
9.35 


6.65 


0.57, 


CONCEPTS 


MM 
RL 


64  -0.74 
30 


-0.80 


11.86 
12.77 


4.85 
5.25 


3.58 


-0.21 


APPLICATION 


MM 
RL 


64 

30 


-0.78  -0.79 


10.86 
11.67 


5.40 
5.97 


3.87 


-0 . 20 


MATH 
TOTAL 


MM 

lEUL 


64 
30 


2.29 


0.81 


51.83 
49.70 


16.2'3 
18.24 


11.04 


0.21 


er|c;. 
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Table  4-A7 

Longitudinal  Analysis  for  GRADE  4  to  GRADE  5. 
PERIOD  -  YEAR  3  to  YEAR  4. 

Ppsttest-  CST  AND  CTBS  Subject  -  READING. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  ■  i-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
77df 


JPosttest 
X  SD 


Residual  Standardized 
SD  T.E. 


PART  A 


MM 
RL 


71 
35 


3.32 


2.23 


39.94 
42.69 


9.37 
6.47 


6.18 


0.54 


PART  B 


MM 
RL 


71 
35 


5.56 


3.10 


36.92 
40.34 


11.22 
8.18 


7.42 


0.75 


READING 
TOTAL 


MM 
RL 


71 
35 


8.88 


3.02 


76.86 
83.03 


19.70- 
13.65 


12.18 


0.73 


CTBS 


Treatment  Treatment  (t) 

Group  N  Effect  75df 


P^osttest 
X  SD' 


Residual  Standardized 
SD  T.E. 


VOCABULARY 


MM 
RL 


72, 
34 


0.62 


0.56 


17.69 
18.09 


7.54 
6.90 


4.39 


0.14 


"COMPREHENSION 


MM 
RL 


72 
34 


1.29 


1.05 


17.49 
17.94 


7.90 
8.29 


4.81 


0.27 


READING 
TOTAL 


MM 
RL 


72 
34 


1.90 


1.00 


35.18 

-^.03 


14.37 
14.38 


7.53 


0.25 


ERJC 


3'^ 
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Table  4-48 

Longitudinal  Analysis  for  GRADE  4  to  GRADE  5. 
PERIOD  •  YEAR  3  to  YEAR  A. 

Posttest-  CST  AND  CTBS  Subject  -  LANGUAGE.  . 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  ■  1-4. 


CST 


Treatment  Treatment  (t) 

Group  N  Effect  77df 


Posttest 
X  SD 


Residual  Standardized 
SD  -T.E.  i 


PART  A 


MM 
RL 


71 
35 


5.24 


3.75 


41.61 
46.03 


7.04 
7.39 


5.79 


0.90  I 


PART  B 


MM. 
RL 


71 
35 


4.43 


3.78 


38.39 
42.09 


7.89 
7.06 


4.85 


0.91 


LANGUAGE 
TOTAL 


MM 
RL 


71 
35 


9.67 


4.73 


80.00 
88.11 


13.48 
12.97 


8.47 


I.IA 


CTBS 


Treatment 

Group  N 


Treatment 
Effect 


(t) 
75df 


posttest 
X  SD 


Residual  Standardized 
SD  T.E. 


SPELLING 


MM 
RL 


72 
34 


0.11 


0.08 


31.83 
30.65 


7.43 
7.95 


5.35 


0.02 


MECHAIUCS  MM 
RL 


72 
34 


0.77 


0.89 


10.90 
11.32 


4.04 
4.01 


3.44 


0.22 


EXPRESSION 


MM 
RL 


72 
34 


0.55 


0.52 


16.10 
15.97 


5.42 
5.74 


4.23 


0.13 


LANGUAGE 
TOTAL 


MM 
RL 


72 
34 


1.44 


0.56 


58.83 
57.94 


14.64 
14.83 


10.26 


0.14 


ERIC 
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Grades  3""5  vs*  cohort  controls*    On  the  next  few  pages  are  the  data 
for  a  three-^ear  longitudinal  study  comparing  CAI  students  with  their 
cohort  controls  in  the  same  schools  one  year  earlier.     Table  4-49 
presents  the  results  for  the  CST  and  CTBS  mathematics  data.     For  the  CST, 
the  treatment  effects  for  the  MM  group  are  all  statistically  significant, 
as  are  2  out  of  3  of  the  effects  for  the  RL  group.     The  RL  group  is  almost 
half  a  standard  deviation  and  the  MM  group  is  more  than  one-and-one-quarter 
standard  deviations  above  the  adjusted  mean  of  the  cohort  controls. 
Discussion  of  the  performance  of  the  RL  group  will  be  covered  in  the 
discussion  section  at  the  end  of  the  presentation  of  results.     For  the 
CTBS  data,  treatment  effects  for  the  MM  group  on  computation  and  mathema^ 
tics  total  are  statistically  significant.     Treatment  effects  on  the 
remaining  subtests  for  the  MM  group  and  all  the  subtests  for  the  RL  group 
are  not  signif ;Lcant. 

Table  4-50  presents  the' reading  results..    For  the  CST  data,  all  the 
treatment  effects  for  the  RL  group  and  the  treatment  effects  for  the 
the  MM  group  on  part  A  are  statistically  .signif icant.     For  the  CTBS 
data,  only  the  treatment  effects  for  vocabulary  are  statistically 
.significant,  and  they- are- roughly  equivalent  for  the  ^IM  and  RL  groups. 

Table  4-51  presents  the  language  results.    For  the  CST  data,  treatment 
effects  on  the  language  totaj.  are  statistically  significant  for  jsoth  the 
RL  and  MM  group.    Adjusted  means  for  the  MM  students  are  almost  half  a 
standard  deviation  above  their  cohort  controls.    Adjusted  means  for  the  / 
RL  students  are  more  than  a  standard  deviation  above  the  cohort  controls 
for  th^  language  CST.     For  the  CTBS  data,  there  are  significant  treatment 
effects  for  both  the  MM  and  RL  groups  in  language  mechanics  and  for  the 
MM  group  in  language  total. 
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Longltudlnal  Analysis 


Table  4-49 
for  GRADE  3  to  GRADE  5:  CAI  VS  COHORTS. 


PERIOD  -  YEAR  2  to  YEAR  4  FOR  CAI  AND  YEAR  1  to"  YEAR  3  FOR  COHORTS. 


Posttest-  CST  AND  CTBS 

Treatment  effects  and 
Schools  -  2-4. 


Subject  -  MATHEMATICS, 
posttest  means  by  treatment  group, 


CST 


PART  A 


Treatment 
Group 


MM 
RL 
COHORT 


Treatment 
N  Effect 


58 
29 
111 


11.88 
4.83 


(t) 
188df 


7.37 
2.30 


posttest 
X  SD 


37.60 
32.41 
25.10 


11.20 
.10.^7 
11.62 


Residual 
SD 


9.85 


Standardized 
T.E. 


1.21 
0.49 


PART  B 


-  MM 
RL 
COHORT 


58 
29 
111 


8.21 
2.57 


6.73 
1.62 


34.69 
30.34 
25.66 


10.90 
8.92 
8.17 


7.45 


1.10 
0.34 


MATH 
TOTAL 


MM 
RL 
COHORT 


58 
.29 
111 


20.09 
7.40 


7.68 
2.17 


72.29 
62 . 7  6 
50.76 


21.08 
18.33 
18.70 


16.00 


1.26 
0.46 


CTBS 


Treatment 
Group 


Treatment 
Effect 


Ct) 
193df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


COMPUTATION 


MM 
RL„ 
.COHORT 


60 
30 
113. 


5.22 
0.58 


4.37 
0.38 


29.43 
25.57 
23.96 


8.55 
9.4.7 
9.  01 


7.41 


0.70 
0.08 


CONCEPTS 


MM 
RL 
COHORT 


60 
30 
113 


0.62 
0.39 


l.iO 
0.53 


II,.  95 
12.43 
11.05 


5.02 
5.24 
4.10 


3.49 


0.18 
O.U 


APPLICATIONS 


MM 
RL 
COHORT 


60 
30 
113 


1.27 
1.02 


1.71 
1.07 


11.05 
11.30 
9.49 


5.70 
5.97 
5.19 


4.59 


0.28 
0.22 


MATH 
TOTAL 


'  MM 
RL 
COHORT 


60 
30 
113 


7.10 
1.99 


3.42 
0.74 


52.43 
49-.  30 
44.50 


16.68 
18.26 
16.22 


12.88 


0.55 
0.15 


ERIC 
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Table  4-50 

Longitudinal  Analysis  for  GRADE  3  to  GRADE  5:  CAI  VS  COHORTS.  ' 

PERrOD  -  YEAR  2 -to  YEAR  4. FOR  CAI  AND  Y^AR  1  to  YEAR  3  FOR  COHORTS. 

Posttest-  CST  AIJD  CTBS  .  Subject  -  READING. 

'Treatment  ef feels  and  posttest  means  by  treatment  group, 
Schools  -  '2-4.  _  ' 


CST. 


Treatment 
Group 


Treatment 
"'  Effect 


(t) 
181df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E.. 


PART  A 


MM 
RL 
COHORT 


55 
30 
108 


2.98 
5.51 


2.12 
3.13 


38.55 
42.07 
35.92 


10.27 
7.20 
11.40 


8.39 


0':35 
0.66 


PART  B 


MM  - 
RL 
COHORT 


55 
30 
108 


1.00 
4.22 


0.60 
2.02 


34.95 
39.00 
34.20 


12.27 
9.61 
12.66 


9.96 


0.10 
0.42 


READING 
TOTAL 


MM 
RL 
COHORT 


55 
30 
108 


3.98 
9.73 


1.38 
2.68 


73.49 
81.07 
70/l2 


21.56 
15.84 
23.38 


17.30 


0.23 
0.56. 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
193df 


P^osttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


VOCABULARY 


MM 
RL 
COHORT 


60 

31" 
114 


1.76 
2.14 


2.11 
2.01 


18.40 
19.42 
16.78 


8.54 
8.56 
7.42 


5.17 


0.34 
0.41 


COMPREHENSION      MM  60  1.07         1.09         18.33  9.37 

RL  31  1.21         0.96         18.84  9.66 

COHORT         114  -  ,  17.44  7.64 


6.12 


0.17 
0.20 


READING 
TOTAL 


MM 
RL 
COHORT 


60 
31 
114 


2.83 
3.35 


1.78 
1. 64 


36.73 
38.26 
34.22 


17.03 
17.40 
14.01 


6.45 


0.44 
0.52 


ERIC 
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Table  4-51 

Longitudinal  Analysis  for  GRADE  3  to  GRADE  5:  CAI  VS  COHORTS. 

t  '  '  '  ■■  -  .  ■  

PERIOD  -  YEAR  2  to  YEAR  4  FOR  CAI  AND  YEAR  1  to  YEAR  3  FOR  COHORTS. 

Posttest-  CST  AND  CTBS  Subject  -  LANGUAGE. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  "2-4. 


CST 


Treatment 
Group 


Treatment  (t) 
Effect  181df 


Posttest 
X  SD 


Residual  Standardized 
SD  x.E. 


PART  A 


MM 
RL 
COHORT 


55 
30 
108 


4.04 
8.10 


2.97 
4.75 


40.80 
45.43 
37.15 


7.81 
8.13 
11.85 


8.14 


0.50 
0.99 


PART  B 


MM 
RL 
COHORT 


55 
30 
108 


1.56 
5.12 


1.55 
4.05 


37.78 
41.77 
36.40 


8.00 
7.43 
8.04 


6.03 


0.26 
0.85 


LANGUAGE 
TOTAL 


MM 
RL 
COHORT 


55 
30 
108 


5.59 
13.22 


2.69 
5.06 


78.58 
87.20 
73.55 


14.27 
14.15 
18.80 


12.47 


0.45 
1.06 


CTBS 


Treatment 
Group 


Treatment  (t) 
Effect-  i93df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


SPELLING 


MM 
RL 
COHORT 


60 
31 
114 


1.28 
1.32 


1.23 
0.99 


31.88 
32.65 
31.06 


8.20 
8.23 
8.69 


6.45 


0.20 
0.20 


MECHANICS 


•  MM 
RL 

■  COHORT 


60 
31 
114 


1.56 
1.6^ 


2.90 
2.36 


11.17 
11.71 
9.80 


4.23 
3.88 
4.45 


3.34 


0.47 
0.49 


EXPRESSION 


MM 
RL 
COHORT 


60 
31 
114 


0.72 
0.79 


0.99 
0.85 


16.23 
16.90 
15.70 


6.29 
6.88 
*6.65 


4.49 


0.16 
0.18 


LANGUAGE 
TOTAL 


MM 
RL 
COHORT 


60 
31 
114 


3.56 
3.73 


2.06 
1.69 


59.28 
61.26 
56.56 


16.61 
15.61 
17.30 


10.70 


0.33 
0.35 


ERIC 
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Grades  2-5  CAI  vs.  comparison  students*    On  the^next  few  pages  are 
the  tables  for  a  four-year  longitudinal  study  of  CAI .  .students  co^  to 
students  in  two  different  schools  in  the  same  years.    Table  4-52  presents 
the  mathematics  results.    For  the  CST  data,  treatment  effects  for  the  MM 
group  are  all  statistically  significant,  while  treatment  effects  for  the 
RL  group  are  not.    The  MM  students  performed  on  the  average  more  than  one 
standard  deviation  above  the  adjusted  mean  for  comparison  students.  For 
the  CTBS  data,  only  the  treatment  effect  for  the  MM  group  on  computation 
is  statistically  significant  and  equivalent  to  .45  of  a  standard  deviation. 

Table  4-53  presents  the. reading  rfesults.    None  of  the  treatments 
effects  is  significant  either  for  the  CST  or  CTBS  data. 

Table  4-54  presents  the  language  results.     For  the  CST  data^  statis- 
tically significant  treatment  effects  were  achieved  by  the  RL  group  in 
part  A  and  the  total  .language  CST.    The  adjusted  mean  for  the  RL  group 
is  .46  of  a  standard  deviation  higher  than  the  adjusted  mean  of  the 
comparison  group.     For  the  CTBS,  none  of  the  treatment  effects  is 
significant. 
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'  ■  -  »■  . 

'  Table  4-52 

Longitudinal  Analysis  for  GRADE  2  to  GRADE  5:  CAI  VS  COMPARISONS. 
PERIOD  -  YEAR  1  to  YEAR  4. 

Posttest-  CST  AND  CTBS  .  Subject  -  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  ■  2-4  vs  5-6. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
157df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


MM 

rL 

comparison 


44 
31 
90 


8r61 
1.57 


5.25 
0.87 


39.05 
31.84 
29.82 


11.00 
10.81 
10.35 


8.44 


1.02 
0.19 


PART  B 


MM 

COMPARISON 


44 
31 
90 


6.57 
0.02 


4.46 
0.01 


36.00 
29.39 
29.00 


10.2-6 
9.43 
8.95 


7.59 


0.87 
0.00 


MATH 
TOTAL 


MM 
RL 

COMPARISON 


44 

31 
90 


15.18 
1.59 


5.24 
0.50 


75.05 
61.23 
58.82 


20.47 
1-9.36 
18.43 


14.92 


1.02 
0.11 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
160df 


Posttest 
X  SD 


Residual  Standardized 
SD   T.E. 


COMPUTATION 


MM 
RL 

COMPARISON 


46 
32 
90 


3.74 
-0.24 


2.34 
-0.13 


29.63- 
25.88 
26.32 


8.82 
8.96 
9.09 


8.26 


0.45 
■0.03 


CONCEPTS 


MM 
RL 

COMPARISON 


46 

32 
90 


-0.13 
-0.08 


-0.15 
■0.09 


12.80 
12.84 
12.72 


5.32 
5.10 
4.95 


4.36 


-0.03 
-0.02 


APPLICATIONS 


MM 
RL 

COMPARISON 


46 
32 

90 


1.36 
0.97 


1.50 
0.97 


11.83 
11.41 
10.27 


6.08 
5.84 
5.22 


4.72 


0.29 
0.21 


MATH 
TOTAL 


MM 
RL 

COMPARISON 


46 
32 

90 


4.97 
0.65 


1.76 
0.21 


54.26 
50.13 
49.31 


17.82 
17.53 
16.81 


14.62 


0.34 
0.04 
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Table  4-53-  ,  _  ■ 

Umgituainal  . Analysis  for  GRADE.  2  to  <?RADE  5;.  CAI  VS  COMPARISONS, 
PERIOD  -  YEAR  1  to  YEAR  4; 

Posttest-  CST  AND  CTBS  Subjecjt  -  READING. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -  2-4  VS  5-6. 


CST 


, Treatment 
Group 


Treatment  (t) 
Effect  150df 


_Posttest 
X        •  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


MM 
RL 

COMPARISON 


43 

30 
85 


-0.27 
0.74 


-0.16 

0.'42 


40.93 

41.93, 

40.01 


9.52 
7.27 
8.64 


8.07 


-0.03 
0.09 


PART  B 


MM- 
RL 

COMPARISON 


43 

30 
85 


-0.06 
1.09 


-0.03 

0,50 


37.35 
38.63 
36.18 


■10.92 
9.83 
10.93 


9.99 


-0.01 
0.11 


READING 
TOTAL 


MM 
RL 

COMPARISON 


43 
30 

85 


-0.33 
1.83 


-0.10 
0.50 


78.28 
80.56 
76.19 


19:26 
16.19 
18.60 


16.92 


-0.02 
0.11 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
160df 


posttest 
X  SD 


Residual  Standardized 
.  SD  T.E. 


VOCABULARY 


MM 
RL 

COMPARISON 


47 
32 
89 


-0.58 
-0.25 


-0.49 
-0.19 


19.30 
19.75 
18.39 


8.49 
8.40 
7.40 


6.16 


-0.09 
-0.04 


COMPREHENSION 


MM 

COMPARISON 


47 
32 
89 


-1.45 
-1.87 


-1.09 
-1.28 


19.30 
19.31 
19.33 


9.50 
9.19 
7.97 


6.96 


■0.21 
-0.27 


READING 
TOTAL 


MM 
RL 

COMPARISON 


47 
32 
89 


-2.02 
-2.12 


-0.89 
-0.85 


38.60 
39.06 
37.72 


17.26 
16.75 
14.21 


11.95 


-0.17 
-0.18 
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Ta&le  4-54 

Longitudinal  Analysis  for  GRADE  2  to  GRADE  5:  qIi  VS  COMPARISONS, 
PERIOD  -YEAR  1  to  YEAR  4.  ^ 

i 

Posttesf  CST  AND  CTBS  Subject  «  LANGUAGE. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  ■  2-A  VS  5-6. 


GST 


Treatment 
Group 


Treatment  (t) 
Effect  150df 


Tosttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


MM 
RL 

COMPARISON 


43 
30 
85 


0.74 
3.66 


0.50 
2.24 


42  ..12 
45.13 
40.24 


6.61 
8.39 
8.98 


7.53 


0.10 
0.49 


PART  B 


MM 
RL 

GOMPARISON 


43 
30 
85 


0.76 
2.65 


0.50 
1.60 


39.21 
41.27 
37.71 


7.26 
7.60 
9.23 


7.63 


0.10 
0.35 


LANGUAGE 
TOTAL 


MM 
RL 

COMPARISON 


43 
30 
85 


1.51 
6.31 


0.55 
2.11 


81.33 
86. 4g 
77.94 


12.54 
14.42 
17.09 


13.79 


0.11 
0.46 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
160df 


posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


SPELLING 


MI'l 
RL 

COMPARISON 


47 
32 
89 


-0.36 
-0.88 


-0.27 
-0.61 


32.94 
32.91 
32.34 


7.93 
7.93 
8.19 


6.58 


-0.05 
■0.13 


MECHANICS 


MM 
RL 

COMPARISON 


47 
32 
89 


0.59 
0.50 


0.77 
0.59 


11.43 
11.53 
10.38 


4.42 
3.88 
4.66 


4.01 


0.15 
0.12 


EXPRESSION 


MM 
RL 

COMPARISON 


47 
32 
89 


-0.18 
0.41 


-0.18 
0.38 


16.58 
17.25 
15.44 


6.21 
6.67 
6.15 


5.18 


'0.03 
0.08 


LANGUAGE 
TOTAL 


MM  . 
RL 

COMPARISON 


47 
32 
89 


0.05 
0.02 


0.02 
0.01 


60.94 
61.69 
58.15 


16.53 
15.06 
16.55 


,  13.17 


0.00 
0.00 


ERIC 
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Dlscussion  for  Grades  2*5 

Mathematics.     The  mathematics  data  for  the  grades  2-5  longitudinal 
group  are  remarkably  consistent.    Table  4-55  presents  an  overview  of  each 
of  the  one*year  studies  and  each  of  the  longitudinal  studies  we  have  just 
seen.     Entries  In  the  table  are  treatment  effects  in  terms  of  residual 
standard  deviations.     Statistically  significant  effects  are  starred.  In 
each  cell  the  first  figure  is  for  the  CST  total,  and  the  remainder  are 
for  the  CTBS  subtests:    computation,  concepts,  and  applications. 

Consistently,  the  CST  figures  indicate  that  the  mathematics  CAI 
curriculum  is  doing' what  it  was  designed  to  do:    viz.,  give  students 
drill-and-practice  in  mathematics  computation.     If  you  compare  results  of 
the  one-year  study  at  grade  4  with  the  two-year  study  ending  at  grade  4, 
results  for  one  .year  are  .63  and  for  two  years  .99  of  a  standard  deviation. 
At  the  fifth  grade  the  figures' for  one,  two,  and  three  years  of  MM  CAI 
are  .76,  1>18  and  1.17  of  a  standard  deviation.     Students  are  learning 
the  mathematics  CAI  curriculum  ^nd  are  demonstrating  superior  performance 
when  tested  on  that  curriculum.     The  treatment  effects  for  the  MM  group 
when  compared  with  cohort  controls  or  'students  in  comparison  schools  also 
demonstrate  superior  performance.  ' 

The  CTBS  computation  figures  also  demonstrate  that  the  CAI  mathematics 
drill-and-practice  curriculum  can  produce  significant  effects  on  a 
standardized  test.    When  the  MM  group  is  contrasted  with  the  RL  group, 
significant  treatment  effects  for  onfe 'year  of  CAI  are  .28  and  .52,  for 
two  years  .59  and  .57,  and  for  three  years  .79  of  a  standard  deviation. 
Students  with  three  years  of  mathematics  CAI  function  at  .70  of  a  standard 
deviation  higher  than  students  in  the  same  schools  without  CAI  and  .45  of 
a  standard  deviation  higher  than  students  in  comparison  schools. 
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Table  4-55 

Treatment  Effects  Given  in  Standard  Deviations  for 
Mathematics  Scores  for  the  Longitudinal  CAI  Cohort,  Grades  2-5 


/ 


Grade  2 

Grade  3 

Grade  4 

Grade  5 

MM  vs  M  ^ 

1  MM  vs  ML'  ^ 

1  MM  vs  RL  ^ 

1  MM  vs  RL  ^ 

'  (N-220f)  ^ 
CST  .08 
COMPUTATION  ^.35* 
CONCEPTS  &  '.33* 
APPLICATIONS 

'  (N«190+)  ^ 
.67* 
.38* 
-.03 
.04  • 

'  (N=265+) 

.63* 
.28* 
.06 
-.10 

'  (N-U5+)  ^ 
.76* 
.52* 
-.10 

f\rsi: 
(COMPUTATION 
CONCEPTS  & 
^PLICATIONS 

J  MM  vs  RL      (N  »  8 

7+) 

—  ^ 

* 

1.17* 
.79* 
-.10 
-.05 

CST 

COMPUTATION 
CONCEPTS  & 
APPLICATIONS 

,  MM  vs  RL      (N  =  134+)  ^ 

r 

.99* 
'.59* 
.21 
.36 

CST 

COMPUTATION 
CONCEPTS  & 
APPLICATIONS 

1  MM  vs  RL    (N  =  98 

r  > 

1.18* 
.57* 
-.21 
-.20 

CST 

COMPUTATION 

CONCEPTS  & 
, APPLICATIONS 

1  MM  &  RL  CAI  VS  Co 

hort  Controls     (N  ■ 

198+)  ^ 

MM  RL' 
1.26*  .46* 
.70*  .08 
.18  .11 
.28  .22 

1  MM  &  RL  CAI  vs  Co 

mparison  School  Stu 

dents     (N  -=  165+) 

CST 

COMPUTATION 
CONCEPTS  & 
APPLICATIONS 

MM  RL 
1.02*  .11 
.45*  -.03 
-.03  -.02 
-.29  .21 

*P<.05 

ERIC 
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with  regard  to  concepts  and  applications,  the  only  statistically 
significant  treatment  effect  is  in  grade  2  when  that  portion  of  the  test 
is-read  to  students.  -In  grades  3-5  when  the  word  problems  of  the  concepts 
and  applications  subtests  are  not  read  to  students,  there  are  no  statisti- 
cally signif ic,ant  treatment  effects.    The  MM  condition  may  help  students 
with  the  mathematics  of  the  problems  while  the  RL  condition  may  be 
helping  with  the  reading.     Some  slight  evidence  for  that  exists  when  CAI 
students  are  compared  to  their  cohort  controls  or  comparison  students. 
In  those  comparisons,^  the  MM  and  RL  students  perform  in  very  similar  ways. 

Reading.    Table  4-56  presents  an  overview  of  the  reading  results 
for  each  of  the  we-year  studies"*^nd  each  of  the  reported  longKudinal 
studies  for  the  grades  2-5  cohort.     Reading  results  are  mixed.     Even  for 
the  CSTs. based  on  the  reading  curriculum,  the  RL  group's  treatment 
effects  are  statistically  significant  in  grade  4  but  not  in  grade  5  of 
the  one-year  studies.     For  the  longitudinal  studies  comparing  the  RL  and 
MM  groups,  one  two-year  study  found  statistically  significant  treatment 
effects  while  the  three-year  study  and  an  earlier  less-appropriate 
two-year  study  did  not.    When  RL  students  are  contrasted  with  their 
cohort  controls,  the  RL  students  score  a  statistically  significant  .56  of 
a  standard  deviation  higher,  but  when  they  are  contrasted  with  students 
in  comparison  schools,  there  is  no  significant  difference. 

For  the  CTBS  data,  the  treatment  effect  for  reading  comprehens  m  in 
the  grade  4  ane-year  study  is  the  only  statistically  significant  treatment 
effect  with  two  exceptions.     Significant  treatment  effects  in  reading 
vocabulary  are  found  for  both  the  RL  and  MM  students  when  they  are 
contrasted  with  their  cohort  controls.     Overall,  the  results  are  unclear. 
Treatment  effects  are  more  likely  to  be  positive  than  negative,  but  no 
strong  pattern  emerges. 
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Table  4-56 

Treatment  Effects  Given  in  Standard  Deviations  for 
Reading  Scores  for  the  Longitudinal  CAT  Cohort,  Grades'  2-5 


Grade  2 

Grade  3 

Grade  4 

Grade  5 

1  MM  vs  M  _ 

j  ML  vs  MM  ^ 

1  RL  vs  MM  - 

1  RL  vs  MM  ^ 

^  N-220+) 
CST 

VOCABULARY 
COMPREHENSION 

N«199+) 

-.15 
-.16 
-.29 

'  N=267+) 

.34* 

.25 

.26* 

'  N=171+) 

.20 
.31 

CST 

VOCABULARY 
COMPREHENSION 

.    RL  vs  MM    (N  «  85+) 

r 

 :  ^ — > 

.31 
.24 

CST 

VOCABULARY 
COMPREHENSION 

,  RL  vs  MM    (N  =  137+)        *    ^  ^ 

> 

.16 
.04 
.02 

CST 

VOCABULARY 
COMPREHENSION 

1  RL  vs  MM    (N  =  10 

p— — > 

.73* 
.14 
.27 

CST 

VOCABULARY 
COMPREHENSION 

1  RL  and  MM  vs  Cohort  Controls  (N  =  193+) 

1 

RL  MM 

.56*  .23 
.  .41*  .34* 

.20  .17 

.   RL  and  MM  vs  Comparison  School  Students    (N  «  158+)  ^ 

1  

CST 

VOCABULARY 
COMPREHENSION 

'5 

RL  MM 
.11  -.02 
-.04  -.09 
-.27      ■  -.21 

31,: 


*P<.05 

ERIC 


Language ♦     Table  4-37  presents  an  overview  of  the  language  results  for 
each  of  th|  one-year  studies  and  each  of  the  reported  longitudinal  studies  for 
the  grades  2-5  cohort.    The  CST  results  are  consistent  and  are  statistically 
significant.     The  language  CAI  curriculum  is  giving  students  drill-and-practic 
on  a  skill  in  which  they  can  then  demonstrate  superior  performance. 

A  demonstration  of  that  skill  on  the  CTBS  subtests  is  much  more  elusive. 
Only  a  one-year  study  at  grade  4  produced  any  statistically  significant 
treatment  effects.    That  cell  also  contains"  the  largest  number  of  students. 

There  appears  to  be  some  consistency  in  the  mechanics  and  expression 
subtests.    Although  the  treatment  effects  are  not  statistically  significant, 
they  are  uniformly  positive  and  sometimes  quite  high.     If  we  look  at  mechanics 
for  instance,  the  three  one-year  studies  show  treatment  effects  of  .10,  .41*, 
and  .21  of  a  standard  deviation,  while  the  three  within-CAI  longitudinal  ' 
studies  show  effects  of  .40,  .34  and  .22  of  a  standard  deviation. 

The  CAI  treatment  consisting  of  one  session  of  reading  and  one  of 
language  contained  the  same  amount  of  CAI  time  as  the  treatment  calling 
for  two  sessions  of  mathematics.    However,  the  RL  group  received  only 
half  as  much  of  th^  CAI  reading  curriculum  or  the  CAI  language  curriculum 
as  MM  students  received  of  ;the  mathematics  curriculum.     Taljle  4-58  gives 
a  comparison  of  the  treatment  effects  for  language  mechanics  and  mathematics 
computation.    Very  roughly,  the  figures  for  the  treatment  effects  in 
language  mechanics  appear  to  be  about  half  the  size  of  the  effects  for 
computation.     This  could  be  related  to  the  amount  of  CAI  experienced  by 
students,  with  20  minutes  per  day  in  mathematics  superior  to  10  minutes 
in  language  for  producing  treatment  effects.     There  has  been  fairly 
consistent  evidence  that  greater  expojsure  produces  greater  results. 
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Table  4-57 

Treatment  Effects  Given  in  Standard  Deviations  for 
Language  Scores  for  the  Longitudinal  CAT  Cohort,  Grades  2-5 


Grade2 

.  Grade  3 

Grade  4 

Grade  5 

1  MM  vs  M  ^ 

1  ML  vs  MM  ^ 

|R1  vs  MM 

1  RL  vs  MM  ^ 

'       N»220f)           ^  \ 
CST 

SPELLING 

MECHANICS  — 
EXPRESSION 

'  (N«199+) 
^                 ^  .46* 
-.12 

.10  . 

.01 

'      (N=i67+)  ^ 
.65* 
.03 
•  .41* 
.  33* 

'  (N=171) 

.56* 
-.09 
.2i 
.  33 

CST 

SPELLING 

MECHANICS 

EXPRESSION 

,  RL  vs  MM  (N  =  85+) 

—  5> 

.71* 

.06 

.40 

-  :2"4 

CST             "  - 
SPELLING 
MECHANICS 
EXPRESSION 

1  RL  vs  MM  (N  =  137+)         "   ' 

.59* 
-.22 
.34 
.15 

CST 

SPELLING 
MECHANICS 
EXPRESSION  - 

jRL  vs  MM  (N  =  106) 

-—  — > 

•  1.14* 
.02 
.22 
.13 

CST 

SPELLING 
MECHANICS 
EXPRESSION  " 

j  RL  and  MM  vs  Co hoi 

•t  Controls  (N  =  193+)  ^ 

RL  MM 

1.06*      .45*  ^ 
.20  .20 
,49  .47 
.18  .16 

1    RL  and  MM, vs  Comparison  School  Students  (N  ».  158+) 

CST 

SPELLING 

MECHANICS 

EXPRESSION 

RL  MM 
.46*  .11 

-.13  -.05 

.12  .15 

.08  -.03 

I  •  ■ 
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Lanj 
Math< 

Tab 

A  Comparison  of  th 
5uage  Mechanics  ( 10 
smatics  Computation 

Grade  3 

le  4-58 

e  Treatment  Effects 
min/day  of  instruct 
(20  min/day  of  inst 

Grade  4 

for 

ion  and 
ruction) 

Grade  5 

One- 

LANC 
MATI 

year  Studies 

^UAGE  MECHANICS 
I  COMPUTATION 

1 —  > 

.10 
.38 

1  5> 

.41* 
.28* 

1  — > 

.21 

52* 

-i 

Lonj 
LAN 

MATI 

^itudinal 

}UAGE  MECHANICS 
(N  »  20  items) 

i  COMPUTATION 
(N  «  48  items) 

1  — 

> 

.40 
.79* 

LANC 
MAT! 

JUAGE  MECHANICS 
i  COMPUTATION 

I 

.34 
59* 

LANGUAGE  MECHANICS 
MATH  COMPUTATION 

1  ■  — 

 5> 

.22 
..57* 

*  P<  .05 

ERIC 
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The  Grade  4  to  Grade  6  Loneitudinal.  Study 

The  grade  4  to. grade  6  longitudinal  study  is  summarized  as  follo>7s: 

Grade  4  Grade  5  Grade  6 

MM   >   MM-  3>.MM 

MRL  ^   MRL   MRL 

RL  >    RL   ^  RL 

In  year  1,  the  fourth  graders  were  randomly  assigned  to  one  of  the  three 

treatment  conditions:     two  .sessions  of  mathematics  CAI  daily  (MM),  one 

session  of  reading  and  one  of  language  daily  (RL),  or  a  combination  of 

one  session  of  mathematics  daily  with  a  second  ses'sion  which  alternated 

between  reading  and  language.     The  same  curriculums  were  kept  for  three 

consecutive  years  of  the  study.    As  it  has  been  pointed  out  in  the 

one-year  study,  there  are  three  levels  of  treatment  conditions  with 

regard  to  mathematics  and  reading  and  language- arts.    The  persistence  of 

the  same  treatment  condition  was  expected  to  magnify  the  difference  in 

the  treatment  efffects. 

One  serious  problem  developed.     Some  non-readers,  non-English-speaking 

or  limited-English-speaking  students  were  randomly  assigned  to  the  RL  or 

MRL  treatments.    At  phe  onset  of  the  CAI  lab  in  year  1,  it  became  obvious 

that  the  pure  reading  and  language  condition  (RL)  was  impossible  for 

them  to  follow  and  to  a  lesser  extent  so  was  the  MRL  condition.    The  CAI 

coordinators  moved  non-English-speaking  students  to  mathematics  but  kept 

doubtful  "students  in    their  original  assignments  for  a  month  to  six 

_  t 

weeks.     If  no  progress  was  made,  they  then  reassigned  students  to  the 
MM  treatment.    The  inability  of  the  reading  and  language  curriculums  to 
adjust  to  the  needs  of  students  of  low  reading  ability  caused  problems 


for  the  research  design.  Let  us  picture  the  grade  A  conditions  as 
follows; 


MM 

MRL 

RL 

Originally  there  were  three  randomly  assigned  groups  of  students. 
Because  of  the  random  assignment  the  groups  were  about  the  same  size  and 
had  students  of  about  the  same  ability  level.    When  non-readers  or 
non-English  speaking  students  were  removed  from  the  RL  group,  the  RL 
group  became  smaller  and  developed  higher  pretest  and  posttest  means  as  a 
result.    Although  such  students  were  reassigned  to  the  MM  group  after 
making.no  progress  in  reading  or  language,  the  computer  logs  showed  them 
as  receiving  mathematics,  reading,  and  lan'guage  CAI,  i.e.,  as  actual  MRL 
students. 

Attrition  affected  this  longitudinal  group  more  heavily  than 
others  because  of  the  implementation  of  the  LAUSD  desegregation  plan. 
All  sixth-grade  students  from  school  1  and  25  students  from  school  3  were 
moved  to  schools  where  the  CAI  program  could  not  be  implemented.  Differen- 
tial attrition  rates  operating  on  the  groups  over  three  years  further 
complicated  the  longitudinal  design  for  grades  4-6.     The  original  fourth- 
grade  pretest  means  of  the  MM  and  MRL  groups  decreased .with  attrition 
over  three  years,  while  the  pretest  means  of  the  RL  group  increased  with 
attrition. 
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Results  for  the  Longitudinal  Data,  Grades  4-6 

Six  sets-  of  analyses  will  be  presented  In  this  section.    We  will 
present  the  following  regressions: 

(1)  grade  4  (posttests)  to  grade  6:    within  CAT 

(2)  grade  4  (posttests)  to  grade  5:.  within  CAT 

(3)  grade  4  (.pretests)  to  grade  5:    within  CAI 

(4)  grade  5  (pretests)  to  grade  6:-    within  CAI 

(5)  CAI  vs.  cohort  controls 

(6)  CAI  vs.  comparison  schools 

'The  first  four  analyses  Involve  only  CAI  students.    The  last  two  analyses 
compare  CAI  students  \Jrlth  cohort  controls  In  their  own  schools  one  year^ 
later  and  with  non-CAI  students  In  comparison  schools. 

Grades  4-6.  'Table  4-59  presents  the  results  for  the  mathematics 
regression  analyses.     For  the  CST  data,  the  treatment  effects  for  the  MM 
and  MRL  groups  are  statistically  significant.     On  the  average,  MM  students  . 
answered  11.36  more  questions  correctly  than  did  the  RL  group  and  are 
almost  one  standard  deviation  above  them  in  CST  performance  in  mathematics. 
For  the  CTBS  data,  statistically  significant  treatment  effects  are  found 
in  computation  and  math  total  for  both  MM  and  MRL  students.     The  MRL.  students 
also  performed  significantly  better  than  the  RL  group  in  applications. 

Table  4-60  presents  the  reading  results.     None  of  the  treatment 
effects  on  the  CST  is  significant  and  only  the  treatment  effect  for  the 
MRL  group  on  the  CTBS  vocabulary  subtest  is  statistically  significant. 
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Table  4-59      ^  . 

Longitudinal  Analysis  for  GRADE  4  to  GRADE  6. 
PERIOD  -  YEAR  1  to  YEAR  3.              '  ' 

Posttest-  CST  AliD  CTBS  (from  CTBS)  •    Subject  -  MATHEMATICS. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -  2-4. 


CST 


Treatment 
Group 


Treatment  (t) 
Effect-  '  62df 


P^osttest 
X  •  SD 


Residual 
SD 


Standardized 
■  T.E. 


PART  A 


MM 

MRL 

RL 


34 
34 

26- 


6.00 
3.71 


2.75 
1.75 


37.85 
35.50 
35.12 


9.75 
10.21 
8.69 


6.89 


0,,87 
0'.54 


PART  B 


MM 

MRL 

RL 


34 
34 
26 


5.36 
5.07 


2.97 
2.89 


27r47 
26.32 
26.12 


9.90 
9.59 
6.15 


5.69 


0.94 
0.89" 


MATH 
TOTAL 


MM 

MRL 

RL 


34 
34 
26 


11.36 
8.78 


3.05 
2.43 


65.32 
61 . 82 
61.23 


18.93 
19.33 
14.03 


11.75 


0.97 
0.75 


CTBS 


Treatment 
Group 


Treatment  (t) 
Effect         62 df 


posttest 
X     '  SD 


Residual 
SD 


Standardized 
T.E. 


COMPUTATIONS 


MM 

MRL 

RL 


32 
35 
27 


3.42 
3.99 


2.02 
2.47 


32.16 
30.34 
32.07 


9.59 
8.83 
8.67 


5.31 


0.64 
0.75 


CONCEPTS 


m 

MRL 
RL 


32 
35 
27 


0.82 
0.47 


,0.88 
0.53 


14.00 
13.11 
15.48 


4.74 
4.71 
3.52 


2.90 


0.28 
0.16 


APPLICATIONS        MM  32  2.14  1.82  12.53  5.65 

MRL  35  2.39  ,       2.13         11.91  6.08 

RL  27  -  13.33  3.56 


3.68 


0.58 
0.65 


MATH 
TOTAL 


MM 

MRL 

RL 


32 
35 
27 


6.37 
6.85 


2.48 
2.80 


58.69 
55.37 
60.89 


18.17 
17.40 
14.32 


8.04 


0.79 
0.85 


ERIC 


3i 


/ 
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Table  4-60 

Longitudinal  Analysis  for  GRADE  4  to  GRADE  6. 
PERIOD  -  YEAR  1,  to  YEAR  3. 

Posttest-  CST  AND  CTBS  (from  CTBS)  Subject  -  READING. 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -  2-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
59df 


posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


m 

MRL 
RL 


30 
34 
29 


4.38 
3.23 


2.07 
1.40 


36.50 
39.79 
42.72 


10.39 
9.44 
7.65 


7.13 


0.61 
0.45 


PART  B 


MM 

MRL 

RL 


30 
34 
29 


0.61 
2.04 


0.30 
0.92 


31.33 
31 . 62 
36.03 


9.95 
12.40 
9.97 


6.88 


0.09 
0.30 


READING 
TOTAL 


MM 

MRL 

RL 


30 
34 
29 


4.99 
5.25 


1.34 
1.30 


67.83 
71  ."41 
78.76 


19.49 
20.95 
15.75 


12.52 


0.40 
0.42' 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
60df 


posttest 
X  SD 


Residual 
SD 


Standardized. 
T.E. 


VOCABULARY 


MM 

mrl' 

RL 


31 
35 
27 


3.34 
2.71 


2.47 
1.79 


21.19 
23.63' 
26.85 


8.42 
8.47 
7.84 


4.65 


0.72 
0.58 


COMPREHENSION 


READING 
TOTAL 


MM 

MRL 

RL 


MM 

MRL 

RL 


31 
35 
27 


31 
35 
27 


-1.00 
-1.22 


2.34 
1.49 


-0.69 
,-0.75 

I 


0.98 
0.56 


24.84 
23.43 
27.78 


.46.03 
47.06 
54.63 


9.71 
10.03 
9.26 


17.32 
17.74 
16.31 


5.01 


8.24 


-0.20 
-0.24 


0.28 
0.18 


I 


Table  4-61  presents  the  language  results.    Treatment  effects  on  the 
CST  totals  are  statistically  significant  for  both  the  RL  and  MRL  groups. 
The  RL  group,  with  almost  7  more  ^questions  correct  on  the  average  than 
the  MM  group,  is  about  three-quarters  of '  r  standard  deviation  above  them 
in  performance  on  the  language  CST.    Although  treatment  effects  on  the 
CTBS  are  all  positive,  none  is  significant. 
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Table  4-61 

Longitudinal  Analysis  for  GRADE  4  to  GRADE  6. 
P£RIOD  -  YEAR  1  to  YEAR  3. 

Posttest-  CST  AND  CTBS  (from  CTBS)  Subject  =  LANGUAGE. 

Treatment  effects  and  posttest  means  by  treatment  group. 
Schools  -  2-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
59df 


P^osttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


MM 

MRL 

RL 


30 
34 
29 


3.25 
4.37 


2.05 
2.53 


39.83 
42.85 
45.90 


7.84 
6.58 
6.12 


5.32 


0.61 
0.82 


PART  B 


MM 

MRL 

RL 


30 
34 
29 


2.01 
2.39 


1.48 
1.62 


34.97 
37.18 
39.97 


7.38 
6.32 
5.12 


4.57 


0.44 
0.52 


LANGUAGE 
TOTAL 


MM 

MRL 

RL 


30 
34 
29 


5.26 
6.76 


1.98 
2.33 


74.80 
80.03 
85.86 


14.36 
12.28 
10.71 


8.94 


0.59 
0.76 


CTBS 


Treatment 
Group 


Treatment 
Effect 


Xt) 
60df 


Posttest 

X      .  SD 


Residual 
SD 


Standardlz^ed 
T.E.  " 


SPELLING 


MM. 
MRL 
RL 


31 
35 
27 


2.02 
1.15 


1.31 
0.66 


33.42 
35.17 
38.04 


9.19 
8.99 
7.94 


5.32 


0.38 
0.22 


MECHANICS 


MM 

MRL 

RL 


31 
35 
27 


0.66 
0.26 


0.76 
0.26 


11.42 
11.97 
13.52 


4.19 
4.32 
2.97 


2.98 


0.22 
0.09 


EXPRESSION 


MM 

MRL 

RL 


31 
35 
27 


0.52 
0.81 


0.47 
0.67 


19.00 
18.97 
22.74 


7.23 
7.75 
6.43 


3.69 


0.14 
0.22 


LANGUAGE 
JOTAL 


MM 

MRL 

RL 


31 
35 
27 


3.19 
2.21 


1.20 
0.74 


63.84 
66.11 
74.30 


18.27 
19.37 
15.14 


9.15 


0.35 
0.24 


ERIC 
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Grades  4-5  (from  CTBS.  grade  4).    Table  4-62  presents' the  mathematics' 
results.     For  the  CST  data,  the  treatment  effects  for  the  MM  and  MRL^ 
groups  are  all  statistically  significant.    With  17  more  questions 
correction  the  average  than  the  RL  group,  the  MM  group  is  about  1.4 
standard  deviations  above  the  RL  group  in  performance  on  the  mathematics 
CST.     On  the  CTBS,  treatment  effects  in  computation  and  math  total 
are  statistically  significant.     The  MM  group  answered  4.15  more  computation 
questions  correctly^chan  the  RL  group  on  the  average  and  is  about 
three-quarters  of  a  standard  deviation  above  them  in  performance  on 
mathematics  computation.  ^  ^  '  . 

Table  4-63  presents  the  reading  results.     None  of  the  treatment 
effects  for  the  CST  or  CTBS  data  is  statistically  significant. 

Table  4-64  presents  the  language  results.     For  the  CST  data*,  treatment 
effects  for  the  RL  group  are  statistically  significant.     The  RL  group  on 
the  average  answered  6,14  more  questions  correctly  than  the  MM  group  and 
is  about  .59  of  a  standard  deviation  above  them  in  performance  on  the 
language  CST.     For  the  CTBS  data,  none  of  the  treatment  effects  is  * 
statistically  significant. 
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Table  4-62 

Longitudinal  Analysis  for  GRADE  4  to  GRADE  5 
PERIOD  -  YEAR  1  TO  YEAR  2 

Posttest-  CST  &  CTBS  (from  CTBS)    .  Subject  -  MATHEMATICS 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools-"  1-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
I33df 


Posttest- 
X  SD 


Residual 
SD 


Standardized 
I.E. 


PART  A 


MM 

MRL 

RL 


56 
54 
50 


9.59 
5.32 


6.61 
3.74 


33.66 
28.50 
27.40 


11.75 
12.41 
10.09 


6.83 


1.40 
0.78 


PART  B 


MM 

MRL 

RL 


56 
54 
50 


7.60 
4.82 


5.57 
-3.61 


30.34 
27.06 
25.24 


9.22 
10.90 
8.26 


6.42 


1.18 
0.75 


MATH 
TOTAL 


MM 

MRL 

RL 


56 
54 
50 


17.19 
10.14 


6.67 
4.01 


64.00 
55.56 
52.64 


20.39 
22.77 
17.46 


12.13 


1.-42 
0.84 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
I37df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


COMPUTATION 


MM 

MRL 

RL 


53 
57 
53 


4.15 
3.29 


3.66 
2.98 


27.15 
24.98 
24.51 


9.00 
9.16 
9.36 


3ij49 


0.76 
0.60 


CONCEPTS 


MM 

MRL 

RL 


53 
57 
53 


0.68 
-0.17 


1.23 
-0.31 


12.43 
11.54 
12.79 


3.75 
4.09 
4.31 


2.67 


0.25 
-0.06 


APPLICATIONS 


MM 

MRL 

RL 


53 
57 
53 


0.43 
1.00 


0.58 
1.39 


9.79 
10.00 
AO. 40 


4.99 
4.88 
5.14 


3.59 


0.12 
0.28 


MATH 
TOTAL 


MM  ' 

MRL 

RL 


53 
57 
53 


5.26 
4.13 


2.97 
2.39 


49.38 
46.53 
47.70 


14.92 
16.45 
16.59 


8.59 


0.61 
0.48 


ERIC 


3  5  i 
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Table  4-63 

Longitudinal  Analysis  for  GRADE  4  to  GRADE  5  " 
PERIOD  -  YEAR  1  TO  YEAR  2 

Posttest-  CST  &  CTBS  (from  CTBS)      Subject  -  READING 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  "  1-4. 

Treatment 
CST  •  Group 


Treatment        (t)         posttest  Residual  Standardized 

Effect  I29df        X         SD  SD  T.E. 


PART  A                MM  53              -  36.75  10.43  6.40 

MRL  54  -0.50  -0.38  36.41  10.80  -0.08 

RL  51  1.98  1.44  42.57  7.42  0.31 

PART  B                MM  53              -  31.53  11.87  7.64 

MRL  54  -1.18  -0.74  30.70  12.98  -0.15 

RL  51  1.01  0.62  37.65  10.57  0.13. 

READING              MM  53              -  68.28  21.56  12.^77 

TOTAL                 MRL  54  -1.68  -0.63  67.11  22.74  -0.13 

RL-  51  2.99  1.09  80.22  17.47  0.23 

Treatment  Treatment        (t)  posttest  Residual  Standardized 

CTBS                  Group  N  Effect  127df  X  SD  SD  -  T.E. 

VOCABULARY         MM  48              -  19.65  6.81  4.39 

MRL  56  0.11  0.13  19.14  8.76  0.03 

RL  51  0.09           0.10  22.02  8.23  0.02 

COMPREHENSION    MM  48               -  21.13  7.94  5.16 

MRL  56  -0.82  -0.77  19.63  9.32  -0.16 

RL  51  -0.20  -0^.18  23.16  9.23  -0.04.. 

READING               MH  48               "  40.77  13.61  7.95  -  . 

TOTAL                 MRL  56  -0.71  -0.43  38.77  17.39  -0.09 

RL  51  -0.11  -0.06  45.18  16.60  -0.01 


356 
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Table  4-64  , 
Longitudinal  Analysis  for  GRADE  4  to  GRADE  5 
PERIOD  -  YEAR  1  TO  YEAR  2 

Posttest-  CST  &  CTBS  (from  CTBS)      Subject  •=  LANGUAGE 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  ■  1-4., 


Treatment 
^  Group 


N 


Treatment 
Effect 


(t) 
I29df 


posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


MM 
MRL 

RL. 


53 
54 
51 


1.84 
3.61 


1.33 
2.54 


39.92 
41.81 
46.63 


9.61 
9.77 
7.76 


6.64 


0.28 
0.54 


MM 

MRL 

RL 


53 
54 
51 


0.01 
2.53 


0.01 
2.09 


37.64 
37.52 
43.22 


7.77 
10.01 
8.49 


5.63 


0.00 
0.45 


LANGUAGE  MM 
TOTAL  MRL 
RL 


53  -  77.57  16.05 

54  -  1.85  0.86  79.33  18.44 
51             6.14           2.77        89.84  15.23 


10.32 


0.18 
0.59 


Treatment 
Group 


Treatment 
Effect 


(t) 
127df 


P^osttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


SPELLING            MM  48         '     -                     -  32.71  7.08  5.17 

MRL  56  -0.31  -0.29  31.89  9.05  -0.06 

RL  51  1.37           1.24  36.39  8.65  0.26 

MECHANICS           MM  48              -  10.65  3.71  2.88 

MRL  56'  0.90           i.52  10.89  4.92  .  0.31 

/             RL  51  0.28           0.45  11.86  3.91  0.10 

EXPRESSION         MM  48              -  17.52  6.29  3.92 

MRL           .  56  -1.13  -1.39  15.96  6.73  .      '  -0.29 

RL  51  -1.57  -1.89  17.96  7.10  '  -0.40 

LANGUAGE             MM  48              -  60.88  14.71  8.78  - 

TOTAL  ■          .     MRL  56  -0.53  -0.29    »  58.75  18.60  ,  -0.06 

RL  51  0.07           0.04   .    66.21  17.74  0.01 

356 
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Gr'ades  4-5  (from  ITBS,  grade  4).    Table  4-65  presents  the  mathematics 
results  when  the  regression  analyses  use'  the  ITBS  data  as  pretests. 
For  the  CST  data,  treatment  effects  for  the  MM  group  are  statistically 
significant.  The  MM  group  answered  15.46  more  questions  correctly  on  the 
average  than  the  RL  group  and  is  more  than  one  standard  deviation  a>ove 
that  group  in  performance  on  the  mathematics  -CST.    Treatmient  effects  for  the  MM 
group  are  also  statistically  s'ignificant  in  CTBS  computation  and  CTBS 
mathematics  total.     In  computation  the  MM  group  answered  3.77  more  questions 
correctly  than^  the  RL  group  and  is  almost  two-thirds  of  a  standard 
deviation  above  them  in  performance.' 

Table  4-66  presents  the  reading  results.    For  the  CST  total,  the 
treatment  effect  for  the  RL  group  is  statistically  significant,  and  that 
group  is  almost  half  a  standard  deviation  above  the  MM  group  in  performance 
on  the  reading  CST.    None  of  the  treatment  effects  for  the  CTBS  data  is 
statistically  significant. 

Table  4-67  presents  the  language  results.    Again  the  RL  group's 
treatment  effect  on  the  CST  is  statistically  significant,  but  the  MRL 
group's  is  not.    On  the  CTBS  only  the  treatment  effect  for  the  RL  group 
in  spelling  is  statistically  significant. 
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Table  4-65 

Longitudinal  Analysis  for  GRADE  4  to  GKI^E  5 
PERIOD  -  YEAR  1  TO  YEAR  2 

Posttest-  CST  &^'CTBS  (from  ITBS)  Subject  -  MATHEMATICS 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  ■  1-4.  , 


Treatment 
Group 


Treatment 
Effect 


(t) 
85df 


posttest 
X  SD 


Residual 
SD 


Standardized 
T.E.  . 


PART  A 


Mil 

MRL 

RL 


45 
49 
46 


8.57 
2.52 


4.67 
1.39 


34.04 
27.86 
27.83 


11.91 
12.30 
10.18 


8.23 


1.04 
0.31 


PART  B 


MM 

MRL 

RL 


45 
49 
46 


6i89 
3.02 


4.47 
1.98 


30.47 
26.63 
25.37 


9.33 
10.87 
8.48 


6.92 


0.99 
0.44 


MATH 
TOTAL 


MM 

MRL 

RL 


45  15.46  4.94  64.51.  20.61 
49             5.54        ^  1.79        54.49  22.58 

46  -  53.20  17.74 


14.05 


1.10 
0.39 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
90df 


posttest 
X  SD 


Residual 
SD' 


Standardized 
T.E. 


COMPUTATION 


MM 

MRL 

RL 


44 
51 
49 


3.77 
1.50' 


2.92 
1.20 


26.80 
24.76 
24.86 


8.89 
8.78 
9.04 


5.97 


0.63 
0.25 


CONCEPTS 


MM 
MRL 


44 
51 
49 


0.57 
-0.40 


0.96 
-0.69 


12.64 
11.51 
12.65 


3.66 
4.10 
4.29 


2.75 


0.21 
-0.15 


APPLICATIONS 


MM 

MRL 

RL 


44 
51 
49 


0.25 
0.46 


0.32 
0.59 


10.00 
10.14 
10.61 


5.00 
5.02 
5.26 


3.68 


0.07 
0.12 


MATH 
TOTAL 


MM 

MRL 

RL 


44 
51 
49 


4.60 
1.56 


2.23 
0.78 


49.43 
46.41 
48.12 


15.19 
16.31 
16.52 


9.53 


0.48 
0.16 


ERIC 
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Table  4-66 

Longitudinal  Analysis  for  GRADE  4  to  GRADE  5 
PERIOD  -  YEAR  .1  TO  YEAR  2 

Posttest-  CST  &  CTBS  (from  ITBS)      Subject  »  READING 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -  1-4.  / 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
79df 


JPosttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


MM 

MRL 

RL 


45 
50 
49 


--0.80 
4.04 


-0.51 
2.49 


36.53 
36.16 
42.57 


10.61 
10.56 
7.56 


7.23 


-0.11 

0;56 


PART  B 


MM 

MRL 

RL 


45 
50 
49 


-2.07. 
3.14 


-1.13 
1.66 


31.60 
30.32 
37.98 


12.07 
12.74 
10.67 


8.44 


-0.25 
0.37 


READING 
TOTAL 


MM 

MRL 

RL 


45 
50 
49 


-2.88 
7.18 


-0.91 
2.22 


68.13 
66.48 
80.55 


21.91 
22.15 
17.74 


14.46 


-0.20 
0.50 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
78df 


JPosttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


VOCABULARY 


MM 

MRL 

RL 


42 

51 
49 


0.37 
0.38 


0.36 
0.37 


18.93 
19.47 
21.57 


6.15 
8.53 
8.16 


4.64 


0.08 
0.08 


COMPREHENSION  MM 
MRL 
RL 


42 
51 
49 


-1.42 
-0.33 


-1.31 
-0.29 


20.93 
'.9.67 
22.92 


8.00 
9.19 
9.04 


4.98 


-0.29 
-0.07 


READING 
TOTAL 


MM 
MRL 
•  RL 


42 
51 
49 


-1.06 
0.05 


-0.59 
0.03 


39.86 
39.14 
44.49 


13.09 
17.09 
16.34 


8.15 


-0.13 
0.01 


ERIC 
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Table  4-67 

Longitudinal  Analysis  for  GRADE  4  to  GRADE  5 
PERIOD  -  YEAR  1  TO  YEAR  2 

Post test"  CST  &  CTBS  (from  ITBS)      Subject  -  LANGUAGE 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  ■  1-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
79df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
I.E. 


PART  A 


MM 

MRL 

RL 


45 
50 
49 


2.00 
4.84 


1.20 
2.83 


39.09 
41.58 
46.41 


10.74 
9.95 
7.84 


7.63 


0.26 
0.63 


PART  B 


MM 

MRL 

RL 


45 
50 
49 


-0.38 
3.99 


-0.27 
2.79" 


37.20 
37.12 
43.31 


8.16 
10.05 
8.68 


6.40 


-0.06 
0.62 


LANGUAGE 
TOTAL 


MM 

MRL 

RL 


45  -  76.29  17.48 

50  1.62  0,62        78.70  18.59 

49  8.83  3.29        89.71  15.52 


12.00 


0.14 
0.74 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
78df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


SPELLING 


MM 

MRL 

RL 


42 
51 
49 


-1.37 
2.46 


-1.17 
2.06 


32.57 
31.69 
36.47 


7.55 
9.07 
8.73 


5.37 


-0.26 
0.46 


MECHANICS 


EXPRESSION 


MM 

MRL 

RL 

MM 

MRL 

RL 


42 
51 
49 

42 
51 
49 


0.89 
1.05 


-1.53 
-0.47 


1.30 
1.51 


-1.83 
-0.55 


10.36 
11.12 
12.08 

17.36 
15.86 
18.35 


3.98 
4.84 
3.93 

6.49 
6.58 
7.26 


r.  3.12 


3.83 


0.29 
0.34 


-0.40 
-0.12 


LANGUAGE 
TOTAL 


MM 

MRL 

RL 


42 
51 
49 


-2.01 
3.04 


-1.01 
1.50 


60.29 
58.67 
66.90 


15.66 
18.28 
18.09 


9.11 


-0.22 
0.33 


ERIC 


360 


•142- 


Grades  5-6 >    Table  4-68  presents  the  mathematics  results  for  the 
CAI  cohort  in  grades  5-6 »    Treatment  effects  on  the  mathematics  CST 
total  are  statistically  significant  for  the  MM  group  but  not  for  the  MRL 
group*     For  the  CTBS  data,  none  of  the  treatment  effects  is  significant. 

Table  4-69  presents  the  reading  results.    None  of  the  treatment 
effects  is  significant. 

Table  4-70  presents  the  language  results.     For  the  CST  total, 
the  treatment  effect  for  the  RL  group  ts  statistically  significant.  For 
the  CTBS  data,  none  of  the  treatment  effects  is  significant. 


^61 


I 
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Table  4-68 

Longitudinal  Analysis  for  GRADE  5  to  GRADE  6 

PERIOD  -  YEAR  2  to  YEAR  3 

t  t  ' 

Posttest-  CST  &  CTBS  '(from  ITBS)      Subject  -  MATHEMATICS 

Treatment  ejects  and  posttest  means  by  treatment  group, 
Schools  -  2-4,. 


Treatment 
Group 


Treatment-' 
Effect 


(t) 
85df 


P^osttest 
X  SD 


Residual 
SU 


Standardized 
T.E. 


PART  A 


MM 

MRL 

RL 


37 

44 
28 


3.22 
2.20 


1.76 
1.27 


39.08 
35.66 
25.21 


9.96 
10.73 
8.67 


6.30 


0.51 
0.35 


PART  B 


MM 

MRL 

RL 


37 
44 
28 


3.99 
3.15 


2.35 
1.96 


29.08 
25.7.0 
25.96 


10.49 
10.44 
7.02 


5.85 


0.68 
0.54 

-  J- 


MATH 
TOTAL 


MM 

MRL 

RL 


37 
44 

28 


7.20 
5.35 


2.24 
1.75 


68.16 
61.37 
61.18 


19.70 
20.70 
14.97 


11.11 


0.65 
0.48 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
94df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


COMPUTATION 


MM 

MRL 

RL 


37 
44 
33 


2.01 
1.62 


1.06 
0.89 


33.19 
30.95 
31.79 


9.71 
8.  73 
8.,32 


7.08 


0.28 
0.23 


CONCEPTS 


MM 

MRL 

RL 


37 
44 
33 


-0.08 
-0.64 


-0.11 
-0.84 


14.14 
12.96 
15.24 


4.9'4 

4yh 
3;  58 


2.97 


-0.03 
-0.22 


APPLICATIONS  MM 
MRL 
RL 


37 
44 
33 


0.80 
1.48 


0.73 
1.41 


13.08 
12.16 
12.94 


5.99 
6.37 
5.50 


4.08 


0.20 
0.36' 


MATH 
TOTAL 


MM 

MRL 
RL 


37 
44 
33 


2.73 

'2.46 


0.91 
0.85 


60.41 
55.82 
59.97 


18.83 
18.06 
14.24 


11.27 


0.24 
0.22 


ERIC 
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Table  4-69 

Longitudinal  Analysis  for  GRADE  5  to  GRADE  6 
PERIOD  -  YEAR  2  TO  YEAR  3 

Posttest-  CST  &  CTBS  (from  ITBS)      Subject  -=  READING 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  ■  2-4. 


CST 


Treatment 
Group 


Treatment 
Effect 


(t) 
79df 


posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


PART  A 


MM 

MRL 

RL 


33 

42- 

30 


0.84 
1.42 


0.47 
0.76 


37.15 
38.41 
41.53- 


11.62 
10.14 
8.29 


6.60 


0.13 
10.21 


PART  B 


MM 

MRL 

RL 


33 

42 
30 


0.34 
2,.  39 


0.19 
1.26 


31.09 
30.79 
34.  $3 


11.38 
tl2.'56 
10.86 


6.73 


0.05 
0.35 


READING 
TOTAL 


MM 

MRL 

RL 


33 
42 

30 


1.18  0.36 
3.80    "  1.13 


68.24 
69.19 
76.47 


21.96 
22.13 
17.59 


11.95 


0.10 
0.32 


CTBS 


Treatment 
Group 


Treatment 
Effect  . 


(t) 
78df 


Posttest 
X  SD 


Residual 
SD 


Standardized 
T.E. 


VOCABULARY 


MM 

MRL 

RL 


35 
40 
31 


2.84 
1.  70 


1.77 
1.00 


21.94 
22.98 
25.84 


8.43 
8.53 
8.94 


5.97 


0.48 
0.28 


COMPREHENSION  MM 
MRL 
■RL 


35  -  25.60  ^  10.10 

40  -0.58^         -0.32        23.65,'  10.63 

31       '.    -1.62         -0.84        26.42  9.72 


6.77 


■0.09 
-0.24 


READING 
TOTAL 


MM 

MRL 

RL 


35  ■  47.54  17.74 

40  .2.26  0.72        46.63  19.48 

31  0.09  0.03       52.26  18.06 


11.73 


0.19 
0.01 


ERIC 
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Table  4-70 

Longitudinal  Analysis  for  GRADE  5  to  GRADE  6 
PERIOD  -  YEAR  2  TO  YEAR  3 

Posttest-  CST  &  CTBS  (from  ITBS)      Subject  -  LANGUAGE 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  -  >2-4. 


Treatment 
Group 


Treatment 
Effect 


(t)  • 
79df 


Posttest 
X  '  SD 


Residual 
SD 


Standardized 
T.iE. 


PART  A 


MM 

MRL 

RL 


33 
42 
30 


0.68 
2.90 


0.47 
1.94 


■40.46' 
42.10 
45.33 


8.33 
7.24 
6.19 


5.31 


0.13 

0.55" 


PART  B 


MM 
RL 


33 
42 
30 


1.14 
2.37 


0.90 
1.79 


35.49 
36.33 
39.53 


7.63 
6.81 
5.41 


4.70 


0.24 
0.50 


LANGUAGE 
TOTAL"^ 


MM 

MRL 

RL 


33 
42 
30 


1.83 
5.27 


0.72 
2.01 


75.94 
78.43 
84.87 


15.41 
13.56 
11.05 


9.27 


0.20 
0.57 


CTBS 


Treatment  Treatment  (t) 

Group  N  Effect  78df 


posttest 
X  SD 


Residual 
SD 


Standardized 
T.E.  ■ 


SPELLING 


MM 

MRL 

RL 


35  -  33.86  9.31 

40  0.78  0.45        34.45  9.87 

31  -0.36         -0.20     ,37.42     ■  8.72 


6.53 


0.12 
-0.06 


MECHANICS 


MM 

MRL 

RL 


35 
40 
31 


0.82 
0.55 


1,00 
0l64 


11.94 
11.90 
13.13 


4.11 
4.15 
3.67 


3.06 


0.27 
0.18 


EXPRESSION 


MM 

MRL 

RL 


35 
40 
31 


0.75 
0.18 


■A- 


0.55 

o.;l2 


19.91 
19.03 
21.71 


7.71 
7.56 
7.11 


5.11 


0.15 
i  0.04 


LANGUAGE 
TOTAL 

I 


1  MM 
MRL 
RL  ■ 


35 
40 
31 


2.35 
0.36 


0.72 
0.11 


65.71 
65.38 
72.26 


18.6.5 
20.05 
17.40 


12.20 


0.19 
0.03 


ERIC 
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Grades  4-6:    CAI  vs  cohort  controls.     Table  4-71  presents  the 
mathematics  results  when  CAI  students  are  compared  to  cohort  contjrols  in 
the**saTne  schools  one  year  later  without  CAI.     For  the  CST  data,  all  of 
the  treatment  effects  are  statistically  significant,  even  those  for  the 
RL  group.    For  the  CTBS  data,  none  of  the  treatment  effects  is  significant. 

f   Table  4-72  presents  the  reading  results.^    None  of  the  treatment 
effects  is  statistically  significant.  ^  — , 

Table  4-73  presents  the  language  results.     For  the  CST  total,  the 
treatment  effects  for  the  RL  and  MRL  groups  are  statistically  significant. 
The  adjusted  mean  score  fc^r  the  RL  group  is  7.75  items  higher, than  the 
adjusted  mean  for  the  cohort  controls,  and  the  RL- group  is  two-thirds  of 
a  standard  deviation  higher  in  performance  on  the  language  CST.     For  the 
CTBS  data,  none  of  the  treatment  effects  is  statistically  significant. 
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Table  4-71 

Longitudinal  Analysis  for  GRADE  4  to  GRADE  6:  CAJ  vs.  COHORTS 
PERIOD  «  YEAR  1  TO  YEAR  3 

Posttest-  CST  &  CTBS  (fromlTBS)      Subject  «  MATHEMATICS 

Treatment  effects  and  posttest  means  by  treatment  group. 
Schools  *  2-4. 


Treatment 

Treatment 

(t) 

Posttest 

Residual 

Standardized 

CST ' 

Group 

N 

Effect 

168df 

X 

SD 

SD 

T.E. 

PART  A 

m 

27 

8.36 

5.09 

38.37 

9.06 

7.44 

1.12 

MRL 

27 

5.52 

3.27 

35.11 

10.11 

0.  74 

IxIj 

3  81 

2.13 

35.61 

8.24 

0.51 

P  AU  AD  T 

in? 

29.10 

8.60 

PART  B 

MM 

27 

6.28 

4.33 

27.48 

9.09 

6.58 

0.95 

MPT 

27 

5.46 

3. 65 

26.26 

9.87 

0.  83 

A,  J 

3  75 

2. 38 

26.52 

6.16 

0.57 

20.67 

7.08 

MATH 

'  MM 

27 

14.64 

5.02 

65.85 

17.56 

13.22 

1.11 

TOTAT 

MRL 

27 

10.98 

3.66 

61.37 

19.45 

0.  83 

RL 

23 

7.56 

2.39 

61.13 

13.50 

0.57 

COHORT 

102 

49.77 

15.08 

Treatment 

Treatment 

(t) 

Posttest 

Residual 

Standardized 

CTBS 

Group 

N 

Effect 

I70df 

X 

SD 

SD 

T.E. 

COMPUTATION 

MM 

25 

0.62 

0.37 

32.00 

9.15 

7.37 

0.08 

MRL 

28 

-1.25 

-0.76 

29.96 

8.87 

-0.17 

RL 

24 

-0.54 

-0.31 

32.13 

8.72 

-0.07 

COHORT 

104 

30.95 

8.64 

CONCEPTS 

m 

25 

0.64 

0.86 

14.44 

4.67 

3.27 

0.20 

MRL 

28 

-1.11 

-1.53 

12.93 

4.86 

-0.34 

RL 

24 

0.21 

0.28 

15.33 

3.59 

0.06 

COHORT 

104 

13.86 

3.92 

APPLICATIONS 

MM 

25 

0.30 

0.32 

13.04 

5.47 

4.14 

0.07 

MRL 

28 

-0.84 

-0.91 

12.04 

6.32 

-0.20 

RL 

24  . 

-1.06 

-1.09 

13.21 

5.86 

.  -0.26 

COHORT 

104 

12.49 

5.28 

MATH 

m 

25 

1.56 

0.57 

59.48 

17.77 

12.10 

0.13 

TOTAL 

MRL 

28 

-3.20 

-1.19 

54.93 

18.29 

-0.26 

RL 

24 

-1.39 

-0.49 

60.67 

14.73 

-0.11 

COHORT 

104 

57.30 

15.71 

ERIC 
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Table  4-72 


Longitudinal  Analysis 

for  GRADE  4  to 

GRADE  6: 

CAI 

vs.  COHORTS 

PERIOD  -  YEAR  1 

TO  YEAR  3 

Posttest"  CST  & 

CTBS 

(from  ITBS) 

Subject 

-  READING 

Treatment  effects  and 

posttest  means 

by  treatment  group » 

Schools  ■  z-^. 

Treatment 

Treatment 

\t ; 

Posttest 

Residual 

Standardized 

CST 

Group 

N 

Effect 

IjDdr 

A 

.  SD 

T  •  £• 

PART  A 

MM 

-2.46 

-1.  ^  / 

40.40 

6.93 

8.31 

-0.30  . 

MRL 

28 

0.72 

0.38 

42.93 

6.20 

u  •  uy 

RL . 

2\17 

1. 

45.61 

6.03 

0.26 

COHORT 

on 

41.34 

7.28 

PART  B 

MM 

25 

0.25 

0. 12 

34.76 

7.32 

8.48 

0..03 

MRL 

28 

0.51 

37.50 

6.29 

0.06 

RL 

28 

2.68 

I.JO 

39.96 

5.21 

0.32 

COHORT 

90 

34.48 

10.09 

READING 

MM 

25 

-2.22 

-0.  62 

75.16 

13.37 

15.26 

-0.15 

TOTAL 

MRL 

28 

1.22 

0.  i5 

80.43 

11.79 

0.08 

RL 

28 

4.85 

1.  JO 

85.57 

10.79 

0.  32 

COHORT 

90 

75.82 

16.18 

Treatment 

Treatment 

(t) 

Posttest 

Residual 

Standardized 

CTBS 

Group 

N 

Effect 

I64df 

X 

SD 

SD 

T.E. 

VOCABULARY  MM 

26 

-2.05 

-1.45 

21.69 

8.02 

6.19 

-0.  33 

MRL 

29 

0.36 

0.26. 

23.79 

7.64 

0.06 

RL 

26 

0.19 

0.13 

26.39 

7.61 

0.03 

COHORT 

98 

23.16 

8.30 

COMPREHENSION  MM 

26 

0.74 

0.49 

25.62 

9.99 

6.61 

0.11 

MRL 

29 

-1.60 

-1.08 

23.17 

9.41 

-0.24 

RL 

26 

-0.33 V 

-0.21 

27.19 

8.94 

-0.05 

COHORT 

98 

23.66 

8.48 

READING 

MM 

26 

-1.32 

-0.50 

47.31 

17.19 

11.55 

-0.11 

TOTAL 

MRL 

29 

-1.24 

-0.48 

46.96 

16.12 

-0.11 

RL 

26 

-0.15 

-0.05 

53.78 

15.70 

-0.01 

COHORT 

98 

46.83 

15.88 

362 
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Table  4-73 

Longitudinal  Analysis  for  GRADE  4  to  GRADE  6:  CAI  vs.  COHORTS 
PERIOD  «  YEAR  1  TO  YEAR  3 

Posttest*  CST  &  QTBS  (from  ITBS)         Subject  «  LANGUAGE 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  "  2-4, 


Treatment 

Treatment 

(t) 

Posttest 

Residual 

Standardized 

CST 

Group 

N  ^ 

Effect 

156df 

X 

SD 

SD 

T.  E. 

- 

PART  A  \ 

25 

-0.83 

-0.62 

36.72 

10.83 

5.71 

-0.14 

MRL 

28 

2.04 

1.55 

40.25 

.  9.09 

n  36 

RL 

28 

3.59  ~" 

2.  73 

42.89 

7.74 

\j  •  \j  ^ 

COHORT 

90 

39.06 

10.03 

PART  B 

MM 

25 

-  0.29 

0.17 

30.96 

10.29 

7.30 

o;o4 

MRL 

28 

3.  58 

2. 14 

31.54 

11.92 

u  •  Hy 

RL 

28 

4.16 

2.  48 

36.11 

10. .14 

U.J/ 

COHORT 

90 

30.14 

11.18 

LANGUAGE 

m 

25 

-0.54  ~ 

-0.20 

67.68 

19.90 

11.75 

-0.05 

TOTAL 

MRL 

28 

5.  61 

■   2. 08 

71.79 

19.96 

n  Aft 

U  .  H  O 

RL 

28 

7.75  ; 

■  2.87 

79.00 

15.  98 

COHORT 

90 

69.47 

19.83 

• 

Treatment 

Treatment 

(t) 

Posttest 

Residual 

Standardized 

CTBS 

Group 

N 

Effect 

164df 

X 

SD 

SD 

T.E. 

SPELLING 

• 

MM 

26 

-0.  66 

-0.49 

34.04 

9.34 

5.91 

-0.11 

MRL 

29 

0.71  ■ 

0.54  . 

35.41 

8.44 

0.12 

RL 

26 

0.66 

.  0.47 

38.31 

7.97 

0.11 

COHORT  V 

98 

35.05 

15.88 

MECHANICS 

MM 

26 

-0.39 

-0.54. 

11.39 

4.12 

3.20 

-0.12 

MRL 

•29 

0.66 

0.92 

11.97 

4.35 

0.21 

RL 

26 

1.01 

1.32 

13.50 

3.03 

0.32 

COHORT 

98 

11.58 

3.87 

EXPRESSION 

MM 

26 

-0.57 

-0.53 

19.34 

7.19 

4.69 

-0.12 

MRL 

29 

-0.49 

-0.47 

19.14 

7.70 

-0.10 

RL 

26 

0.58 

0.52 

22.31 

6.16 

0.12 

COHORT 

98 

19.16 

6.08 

LANGUAGE 

MM 

26 

-1.6? 

-0.  66 

64.77 

17.82 

10.77 

-0.15 

TOTAL 

MRL 

29 

o;88 

0.37 

66.52 

18.65 

0.08 

RL 

26 

2.24 

0.88  - 

74.12 

15.40 

0.  21 

COHORT 

98 

65.80 

14.98 

ERLC 

368 

-150- 


Grades  CAI  vs  comparisons*    Table  4-74  presents  the  mathematics 

results  when  CAI  students  are-  compared  with  students  in  Schools  5  and  6 
without  CAI.    There  are  no  statistically  significant  treatment  effects, 
although  one  can  see  in  the  CST  data  the  expected  pattern  of  effects, 
within  the  three  CAI  groups. 

- ■      Table  4-75  presents  the  reading  results.    Only  the  negative  treatment 
effect  for  the  MM  group  on  part  A  of  the  CST  is  statistically  significant. 
For  the  CTBS  data,  treatment  effects  for  the  RL  and  MRL  groups  in  vocabulary 
are  strongly  positive  although  not  significant. 

Table  4-76  presents  the  language  results.    For  the  CST  data,  treatment 
effects  for  the  RL  group  are  statistically  significant.    None  of  the 
treatment  effects  for  subtests  of  the  CTBS  is  significant. 
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Table  A-74 

Longitudinal  Analysis  for  GRADE  A.  to  GRADE  6:    CAI  vs COMPARISONS 

PBRIOD  «  YEAR  1  TO  YEAR -3 

.   Posttest*  CST  &  CTBS  (from  ITBS)  Subject  « 'MATHEMATICS 

Treatment  effects  and  posttest  means  by  treatment  group, 
Schools  "  2-A  vs.  5-6,.         .  ^ 


CST 

Treatment 
Grbup 

N 

Treatment 
Effect 

(t) 

163df 

Posttest 
X  SD 

Residual 
SD 

Standardized 
T.E. 

PART  A 

MM 

30. 

3.09 

1.77 

38  SO 

Q  f^O 

7  .  U  U 

7.7A 

0 

.39 

■It 

MRL  ' 

32 

2.38 

1.29 

0 

.31 

RL 

25 

^1.10 

-0.  58 

35. AO 

8.A2 

-0 

.1-4 

COMPARISON 

85 

35.  A5 

10.  AO 

PART  B 

MM 

30 

'  0.94 

0.56 

27.87 

10.02 

'7. A3 

0 

.13 

MRL 

32 

1.19 

0.67 

26.69 

9.  A3 

■  * 

0 

.16 

RL 

25 

-1.74 

-0.95 

26.20 

6.20 

-0 

.23  , 

COMPARISON 

85 

• 

26. 85 

10.  26 

MATH 

MM 

30 

4.03 

1.24 

66.37 

19.10 

14.40 

0 

.28 

TOTAL 

MRL 

32 

3.57  .. 

1.04 

63.09 

0 

.  25 

RL 

25 

-2.85 

-0.80  ^ 

61.60 

13.79 

-0 

.20 

COMPARISON 

85 

f 

62.29 

20.18 

Treatment 

0 

Treatment 

(t) 

Postt 

est 

Residual 

Standardized 

CTBS 

Group 

N 

Effect 

162df 

X 

SD 

SD 

T 

E. 

^COMPUTATION 

MM 

28 

-1.42 

-0.79 

32'.  46 

9.54 

7.67 

-0. 

19,. 

MRL 

33 

-2.16 

-1.21 

30.76 

8.42 

-0. 

28 

RL 

26 

-2.46 

-1.32 

32.27 

8.40 

-0. 

32 

COMPARISON 

84 

34.56 

9.61 

CONCEPTS 

MM 

28 

0.30 

0.39 

14.89 

5.03 

3.38 

0. 

09 

MRL 

33 

-0.84  '■ 

-1.07  ^ 

13.18 

4.70 

-0. 

25 

RL 

26 

'0.47 

0.58 

15.50 

.  3.52 

0. 

14 

COMPARISON 

•  84 

14.99 

5.10 

AlPPUCATIONS 

MM 

28 

0.19 

0.17 

13.43 

5.89 

4.93 

0. 

04 

MRL 

33 

0.28 

0.24 

12.79. 

6.33 

0. 

06 

RL 

26 

,-0.53 

-0.44 

13.27 

5.80 

■-0. 

11 

COMPARISON 

84 

13.21 

7.01 

» 

^.MATH 

MM 

28 

-0.92 

-0.30  . 

60.7,9 

19.15 

.  13.07 

-0. 

07 

TOTAL 

MRL 

33 

.  -2.73 

-0.89  ■ 

56.73' 

17.65 

••'-0. 

21  . 

RL  ^ 

26 

.-2.51 

-0.79 

61.0,4-. 

14.29 

-0. 

19 

'.COMPARISON 

84  ' 

,62.76 

.19.40 

EMC 

0 

3 

70 

5 
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Table  4-75 

Longitudinal  Analysis  for  GRADE  4  to  GRADE  6:  CAI  vs.  COMPARISONS 
PERIOD  »  YEAR  1  TO  YEAR  3 

Posttest*  GST  &  CTBS  (from- ITBS)  Subject  «  READING 

Treatment  effects  and  posttest  means  by  treatment  group. 
Schools  «  2-4  vs.  5-6,  - 


GST 


Treatment 
Group 


Treatment 
Effect 


(t) 
170df 


?osttest 
X  SD 


Re  s  idua 1   S  tandar dized 
SD  T.E. 


PART  A 


MM 

28 

-4.39 

-2.60 

37.07 

10.74 

7.22 

MRL 

33 

-0.69 

-0.42 

40.42 

8.88 

RL 

30 

0.85 

0.50 

43.17 

7.55 

COMPARISON 

92 

40.77 

8.10 

-0.61 
-0.10 
0.12 


PART  B 


MM 

28 

-0.18 

-0.10 

31.61 

10.75 

MRL 

33 

0.64 

0.35 

31.82 

11.69 

RL 

30 

2.79 

1.50 

36.57 

9.79 

COMPARISON 

92 

32.05 

10.99 

7.95 


-0.02 
0.08 
0.35 


READING 
TOTAL 


MM 

MRL 

RL 

COMPARISON 


28 

33 
30  . 
92 


-4.57 
-0.05 
3.65 


-1.39 
-0.02 
1.11 


68.68 
72.24 
79.73 
72.83 


20.76 
19.68 
15.68 
18.30 


13.99 


-0.33 
0.00 
0.26 


CTBS 


Treatment 
Group 


Treatment 
Effect 


(t) 
17ldf 


_Posttest 

X  SD 


Residual- Standardized 
SD  '  t%E. 


VOCABULA]^ 

I 


READING 
TOTAL 


ERJC 


MM'' 

29 

-0.06 

-0.05 

22.17 

8.89 

5.97 

■  -0.01 

MRL 

34 

2.32  ' 

1.70 

23.62 

7.99 

0.39 

RL 

28 

2.61 

1.83 

26.89 

7.70 

0.44 

COMPARISON 

93 

22.71 

8.86 

MM 

29  0.96 

0.60 

26.31 

r  10.64 

6.86 

0.14 

.MRL  " 

34 

-0.81 

-0.52 

23.74 

9.95 

-0.12 

RL 

28 

0.16 

0.10 

27.50 

8.81  . 

0.02 

COMPARISON 

93 

24.82 

10.25 

Mtl 

29 

0.90 

0.33 

48.48 

18.83 

11.82 

0.08 

MRL 

34 

■1.51 

0.56  ' 

"  47.35 

17.09 

6.13 

RL  ..  . 

28 

2.77 

0.98 

54.39 

15.69 

0.23 

-COMPARISON 

93  , 

47.53 

18.51 

371 
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Table  4-76 

Longitudinal  Analysis  for  GRADE  4  to  GRADE  6:  CAI  vs^  COMPARISONS 
PERIOD  -  YEAR  1  TO  YEAR  3 

Posttest-  CST  &  CTBS  (from  ITBS)  Subject  »  LANGUAGE 

Treatment  effects  and.  posttest  means  by  treatment  group, 
Schools  "  2-4  vs.  5-6. 


Treatment 

Treatment 

(t) 

'  Postt 

est 

Residual 

Standardized 

CST 

Group 

N 

Effect 

I70df 

X 

SD 

SD 

■  T. 

E. 

PART  A 

MM 

28 

-'0.33 

-0.27 

41.04 

7.29 

5.11 

-0. 

06 

MRL 

.  33 

2.26 

1.91 

43.00 

6.18 

0. 

44 

RL 

30 

3.60 

3.01 

'  46.00 

6.  07 

0. 

70 

COMPARISON 

92 

41.40 

6  76 

PART  B 

MM 

28 

-1.57 

-1.39 

35.79 

i.n 

4.83 

-0. 

32 

MRL 

33 

1.14 

1.02  , 

37.64 

6.39 

0. 

24 

RL 

30 

2.65 

2.34 

40.33 

5.48 

0. 

55 

m 

COMPARISON 

92 

36.83 

6.15 

LANGUAGE 

MM 

28 

-1.89 

-0.90 

76.82 

14.26 

8,99 

-Q- 

21 

TOTAL 

MRL 

33 

3.40 

1.64 

80.64 

11.82 

0. 

38 

RL  • 

30 

6.25 

2.97 

86.33 

11.11 

0. 

70 

COMPARISON 

92 

78.23 

12.15 

... 

Treatment 

Treatment 

(t) 

£ostt 

est 

Residual 

Standardized 

CTBS 

Group 

N 

Effect 

I7ldf 

X 

SD 

SD 

T. 

E. 

MM 

29 

-1.03 

.-0.65 

34.  79 

9.05 

6.82 

-0. 

15 

MRL 

34 

-0.27 

-0.17 

35.  71 

8.49 

-0.' 

04 

RL. 

28 

0.55 

0.34 

38.21 

8.14 

0. 

08 

COMPARISON 

93 

36.24 

9-11 

MECHANICS 

MM 

29 

-Q.69 

-0.93 

11.69 

4.53 

3.22 

-0. 

21 

MRL 

34 

0.51 

0.69 

12.21 

'4.44 

0. 

16 

.  RL 

28 

0.86 

1.12 

13.71 

3.09 

0. 

27 

COMPARISON 

93 

12,.  77 

4.02 

EXPRESSION 

MM 

29 

-0.56 

-0.49 

19.93 

7.91 

4.92 

-0. 

11 

34 

-0.35 

-0.31 

19.44 

7.84 

-0. 

07 

RL 

28 

0.60 

0.51 

22.50 

6.07 

o; 

12    ^  • 

'  COMPARISON 

93 

19.96 

6.61 

LANGUAGE 

m 

29 

-2.29 

-0.80 

66.41 

18.90 

12.36 

-0. 

18 

TOTAL 

MRL 

34 

-0.10 

-0.04 

67.35 

19.09 

>  0. 

01  . 

RL. 

28 

2.01 

0.68 

7  4.  if  3 

15.53 

0. 

16 

COMPARISON 

93 

68.97 

17.63 

ERIC 
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Discussion  for  Grades  4-6 

Discussion  of  the  data  from  the  fourth-  through  sixth-grade  longi- 
tudinal study  will  focus  on  the  three  curriculum  areas:    mathematics » 
residing »  and  language  arts* 

Mathematics*    Table  4-77  presents  ah  overview  of  the  CAI  treatment 
effects  in  standard-deviation  units  for  mathematics.  One-year 
studies  are  presented  in  the  top  row  followed  by  the  six  longTtudinal 
studies  which  have  been  reviewed.    Wherever  the  treatment  effect  is 
followed  by  an  asterisk,  the^effect  is    statistically  significant. 

The  CAI  treatment  effects  are  shown  consistently  by  the  curriculum- 
specific  tests.    Except  in  relation  to  the  comparison  schools,  the  MM 
group  consistently  performed  anywhere  from  two-thirds  of  a  standard 
deviation  to  more  than  a  standard  deviation  above  the  level  of  the  RL 
group.     Clea^rly,  the  drill-and-practice  mathematics  CAI  curriculum  is 
capable  of  increasing  students*  skills  in  mathematical  computation. 

The  data  from  the  CTBS  are  less  striking.     The  within-CAl  analyses 
show  the  MM  group  consistently  superior  in  computation  to  the  RL  group 
with  which  it  is  being  compared.     In  most  cases,  the  performance  of  the  MM 
group, in  computation  is  close  to  two-thirds  of  a  standard  deviation  above 
that  of  the  RL  group  and  is  statistically  significant.    The  MM  group  also 
tends  to  show  improved  performance  in  mathematics  concepts  and  applications 
over  time,  espe^cially  in  the  first  of  the  longitudinal  studies. 

The  fourth-  through  sixth-grade  study  is  not  consistent  with  regard  to 
the  relationship  between  the  amount  of  mathematics  CAI  and  test  performance. 
In  the  one-year  studies  at  grades  5  and-  6,  the  CST  dat|i  and  CTBS  computation 
data  would  seem  to  indicate  that  students  with  two  sessions  of  mathematics 


-155- 


Table  4-77 

Treatment  Effects  Given  in  Standard  Deviations  for  Mathematics 
Scores  for  the  Longitudinal  CAI  Cohort,  Grades  4-6. 


One-Year  Studies 

 ^ 

CST 

COMPUTATION 

CONCEPTS 

APPLICATIONS 


Longitudinal 

CST   " 

COMPUTATION 

CONCEPTS 

APPLICATIONS 


1 


Grade  4 


Grade  5 


MM  6r  MRL  vs>  RL, 


&  MRL  vs.  RL 


"mm 

MRL 

•|  MM 

.90* 

.33 

■  .78* 

-.01 

-.27 

.37* 

-.33 

-.25 

-.02 

-.23 

-.31 

1  -.09 

MRL 
.38* 
.19 
-.27 
.04 


MM  &  MRL  vs.  RL 


Grade  6 


MM  &  MRL  vs.  RL^ 


MM 
.83* 
.33 
.05 
.26 


-  MRL 
.48* 
.16 
-.09 
.21 


.97* 
.64* 
.28 
.58 


.75*' 
.75* 
.16 
.65* 


CST 

COMPUTATION 

CONCEPTS 

APPLICATIONS 


1.42*  .84* 

.76  .60 

.25  -.06 

.12  .28 


CST 

COMPUTATION 

CONCEPTS 

APPLICATIONS 


I  MM  &  MRL  vs.  RL 


1.10* 
.63* 
.21 


.39* 
.25 
-.15 


CST 

COMPUTATION 
CONCEPTS  - 
APPLICATIONS 


,  MM  &  MRL  vs.  RL  • 


.65* 

.48 

.28 

.23 

-.03 

-.22 

.20 

.36 

C3T 

COMPUTATIO"N 

CONCEPTS 

APPLICATIONS 


|MM,  MRL  &.  RL  vs.  Cohort  Controls 
l~  '  F  MM 


m 
i.ii* 

.08 
.20 
.07 


MRL 
.83* 
-.17 
-.34 
-.20 


RL- 
.57* 

-.07 
.06 

-.26 


CST 

COMPUTATION 

CONCEPTS 

APPLICATIONS 

*p  <  .05 


MM,  MRL  &  RL  vs.  Comparison  Schools 


■MM 
.28 
-.19" 
•  .09 
.04 


MRL 
"775 
0  -.28 
-.25 
.06 


RL* 

,20 
,32 
.14 
.11^- 


ERIC 
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CAI  daily  have  treatment  effects  about  twice  the  size  of  those  achieved 
by  students  with  only  one  session.     In  the  four  longitudinal  studies 
involving  only  CAI  students,  the  MRL  group  is  better  than  the  MM  group, 
roughly  equivalent,  and  markedly  or  slightly  inferior.    The  inconsistency 
of  the  relationship  reflects  in  part  the  inconsistencies  in  the  grade  4 
data.     There  are  several  problems.     The  problem  of  randomization  has 
.already  been  discussed.    Another  difficulty  with  regard  to  mathematics 
is  the  character  of  the  ITBS  mathematics  pretest.     It  is  as  much  a  test  " 
of  reading  as  of  mathematics  for  many  of  the  students  in  the  study. 
There  is  no  strictly  computational  component.    There  are  surprisingly 
high  mathematics  ITBS  scores  for  some  Hispanic  students,  and  the  possibility 
that  questions  were  read  in  Spanish  to  bilingual  students  in  some  classes 
cannot  be  overlooked.     The  overall  ITBS  scores  in  grade  4  were  especially 
high  in  year  1.    Added  to  those  complications  were  differential  attrition 
rates  over  time,  with  brighter  students  leaving  the  MM  and  MRL  groups  and 
slower  students  leaving  the  RL  group.    Given  the  inconsistencies,  some 
average  of  the  treatment  effects  for  the  MRL  group  would  give  the  best 
estimate  of  the  effect  of  the  MRL  treatment.     That  would  place  the 
MRL  group  between  one"third  and  one-half  of  a  standard  deviation  above  the 
RL  group  but  still  below  the  MM  group. 

The  cohort  control  and  comparison  school  regressions  are  also 
confounded  because  of  the  problems  with  the  grade  4  data.    Pat-terns  of 
superiority  of  the  MM  group  over  the  RL  group  on  mathematics  achievement 
remain,  but  except  for  the  CST  data  for  CAI  students  vs.  the  cohort 
controls,  none  of  the  treatment  effects  is  significant.    We  will  examine 
the  cohort  control  data  more  closely  inva  later  discussion. 
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Reading,     Table  4-78  presents  an  overview  of  the  CAI  treatment 
effects  in  terms  of  standard  deviations  for  the  grades  4-6  longitudinal 
study  of  reading.     For  the  RL  group,  only  one  of  the  one-year  studies  and 
one  of  the  longitudinal  studies  report  statistically  significant  treatment 
effects  on  the  CSTs.     For  the  MRL  group  there  is  only  one  significant  . 
treatment  effect  for  the  CST  in  the  grade  4  one-year  study.     In  all 
cases  the  RL  group's  performance  is  superior  to  the  MM  group's  on  the 
reading  CiST,  but  there  is  not  the  striking  effect  found,  for  example,  on 
the  mathematics  CST. 

There  is  some  indication  at  the  sixth-grade  level  that  RL  and,  more 
specifically,  MRL  students  performed  better  on  the  CTBS  vocabulary  subtest 
than  did  MM  students.     Whether  one  uses  as  pretests  the  CTBS  from  grade  4, 
the  ITBS  from  grade  5  or  the  ITBS  from  grade  6,  the  CTBS  data  for  grade  6 
show  significant  treatment  effects  for  the  RL  group.     It  is  strange, 
however,  that  the  effects  do  not  show  up  at  grade  5  when  the  same  posttest 
was  administered. 

The  CTBS  compfehj^nsion  subtest  shows  no  Significant  treatment 
effects.  More  often  than  not  the  within-CAI  treatment  effects  for 
comprehension  are  negative'. 

Overall,  no  strong,  sure,  longitudinal  patterns  emerge  from  the 
reading  results  of  the  fourth-  through  sixth-grade  longitudinal  study. 
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Table  4-78 

Treatment  Effects  Given  in  Standard  Deviations  for  Reading 
Scores  for  the  Longitudinal  CAI  Cohort,  Grades  4-6. 


Grade  4 


Grade  5 


Grade  6 


^1 
MRU 


One-Year  Studies 
 > 

CST 


RL  &  MRL  vs. 


RL 


MRL 


.93*  .36* 


VOCABULARY 
COMPREHENSION 

.28 
.23 

.04 
-.04 

-.03 
-.09 

.03 
-.25 

.45* 
.03 

.51*  ' 
-.06 

'-Longitudinal 

EL  & 

MRL  VS. 

MM 

CST  1 

VOCABULARY 

COMPREHENSION 

T 

0.42. 
0.58 
-0.24 

0.40-^ 
0.72* 
-0.20 

RL  &  MRL  vs.  MM 


RL  &  MRL  V8 


RL 
.22 


MRL 
-.10 


RL 
.44 


.18 


iCST 

VOCABULARY 
COMPREHENSION 


RL  & 


MRL  vs.  MM 


.23  -.13 
.02  .03 
,-.04  -.16 


'CST 

VOCABULARY 
COMPREHENSION 


|RL  &  MRL  vs.  MM 


.50*  -.20 
^'  .08  .08 
-.07  -.29 


ICST 

VOCABULARY 
COMPREHENSION 


RL  &  MRL   vs.  MM 


.32 
.28 
-.24 


.lO' 

.48 

.09 


CST 

'VOCABULARY 
COMPREHENSION 


RL,  MRL  &  MM  vs 


.  Cohort  Controls 


*p  <  .05 
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Language.    Table  A-79  presents  an  overview  of  the  treatment  effects 
in  language  for  the  fourth-  thorough  sixth-grade  longitudinal  cohort.  In 
contrast  to  the  CST  results  ^in  reading,  the  language  CST  results  are 
positive,  consistent,  and,  fo^  the  RL  group,  all  statistically  significant. 
The  CAI  language  arts  curriculum  is  clearly  teaching  something  demonstrable 
.and  is  increasing  students'  skills  in  specific  aspects  of  language.  In 
all  cases,  the  MRL  grdup  has  treatment  effects  on  the  CST  above  those  of 
the  MM  group  and  below  those  of  the  RL  group,  as  might  be  expected. 

The  CTBS  data  are,  again,  less  striking  than  the  CST  data.    If  we 
concentrate  on  the  RL  group's  performance,  we  see  several  positive  and  one 
statistically  significant  treatment  effect  in  spelling.    For  mechanics, 
all  of  the  treatment  effects  are  positive,*  although  none  is  statistically 
significant.     There  are  mi?^ed  results  for  language  expression.  When 
contrasted  to  the  cohort  controls  or  to  comparison  school  students,  the 
RL  students  performed  better  on  all  language  tests  but  especially  on 
language  mechanics. 

Overall,  results  on  language  variables  are  more  positive  than  the 
results  for  reading  and  less  positive  than  the  results  for  mathematics. 
Since  the  RL  group  had  only  10  minutes  per  day  in  the  language  CAI 
treatment  compared  to  20  minutes  in  mathematics  for  the  MM  group,  it 
should  not  be  surprising  that  treatment  effects  are  weaker. 
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.  Table  4-79 

Treatment  Effects  Given  in  Standard  Deviations  for, Language 
*    '  Scores  for  the  Longitudinal  CAI  Cohort,  Grades  4-6. 


Grade  4 

Grade  5 

Grade  6 

One-Year  Studie 
 •  5> 

_  RL  &  MRL  VS.  JnW^ 

1  PT    H.  MTJT    vo  MM 

IRL  &  MRL  vs.  MM 

'         RL  MRL 

RL  MRL 
.774*  .16 
.22  -.16 
.16  .16 
-.38  -.25 

'         RL  MKL 

CST 

SPELLING 

MECHANICS 

EXPRESSION 

1.09*  .42* 
..34  .08 
.21  -.01 
.21  -.07 

.53*  .25 
-.03  -.13* 
.21  .07 
.20  .17 

Longitudinal 

CST— ^  

SPELLING  / 

MECHANICS 

fiXPRESSION 

1  RL  &  MB 

Lvs.  MM 

.76*  .59^ 
.22  .38 
.09'  .22 
.22  .14 

CST 

SPELLING 

MECHANICS 

EXPRESSION 

1  RL  &  MR 

L  vs.  MM 

.59*  .18 
.26  -.06 
.10  .31 
-.40  -.29 

CST 

SPELLING 

MECHANICS 

EXPRESSION 

RL  &  MRL  vs.  MM  ^ 

.74*  .14 
.46*  -.26 
.34  .29 
-.12  -.40 

CST 

SPELLING 

MECHANICS. 

EXPRESSION 

j-  RL  &  MRL  vs.  MM 

RL  MRL^ 

.57*  .2b 
-.06  .12 
.18  .27 
.04  .15 

CST 

SPELLING 

MECHANICS 

EXPRESSION 

RL,  MRL  &  MM  vs.  Cohort  Controls 

1 

RL 

.66* 

.11 

.32 

.12 

MRL  MM^ 
.48*  -.05 
.12  -.11 
.21  -.12 
-.10  -.12 

CST  ^ 

SPELLING 

MECHANICS 

EXPRESSION 

RL,  MRL  &  MM  vs.  Comparison  Schools 

1  ■  ;  

i 
1 

RL 

.70* 

.08 

.27 

.12 

MRL  MM^ 
.38  -.21 

-.04  -.15 
.16  -.21 

-.07  -.11 

*p  <  .05 
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Chapter  V 

TREATMENT  EFFECTS:    THE  BIG  PICTURE 

In  this  chapter  the  findings  of  the  one^-year  studies  reported  in 
Chapter  III  and  the  longitudinal  studies  reported  in  Chapter  IV  will  be 
consolidated*     The  one*year  studies  will  be  examined  first,  followed  by 
the  longitudinal  studies*     Finally,  CAI  students  will  be  examined  in 
contrast  with  cohort  cqntrols  and  comparison  students* 

One-Year  Studies 

In  this  section  of  the  report  treatment  effects  will  be  discussed—not 
in  terms  of  numbers  of  items  as  in  earlier  tables  (4-2  to  4-76)— but  in 
terms  of  standard  deviations  as  in  Tables  4-77  to  4-79*     Table  4-80 
presents  an  overview  of  all  the  CAI  treatment  effects  in  the  12  one-year 
studies  arranged  so  that  overall  patterns  may  be  seen*    Entries  can  be 
interpreted  to  show  how  far  above  (+)  or  below  .(-)  the  adjusted  mean  of 
the  control  group  lies  the  adjussted  mean  for  the  treatment  group*  The 
first  entry,  1*16*,  indicates  that  the  M  group,  receiving  one  session  of 
mathematics  CAI  daily,  performed  1*16  standard  deviations  above  the 
control  group  on  the  mathematics  curriculum-specific  test  (CST)*  The 
asterisk  indicates  that  the  treatment  effect  was  statistically  significant* 
In  all  cases  the  standard  deviation  used  is  the^  residual  standard  deviation 
which  appeared  in  the  last  column  of  Tables  4-2  to  4-76*     Our  discussion 
of  the  one-year  studies  will  be  presented  in  three  sections j  one  for 
each  of  the  three  curriculums:    mathematics,  reading  and  language* 

Mathematics 

The  mathematics  CSTs  were  developed  to  test  whether  the  CAI  curriculum 
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Table  *-80 

TreatMnt  Effects  In  Stundiird  Devi nt tons ;     All  One  YeAr  Studies 


MATHEMATICS 

READING 

LANGUAGE 

Trentnient 
Condttlona  Grade 

Year 

Group 

Math. 

CST 

Coffiputatlon 

Concept*  Application 

Math. 
Total 

Group 

Read. 

CST 

Voca. 

Coap. 

Read- 
Total 

Group 

Iji  no  J 

CST 

Spelling 

Mechanics 

Liing. 

Expreaaion  Total 

K'v  ,  0  I 

2 

M 

1.16* 

.12 

.37* 

-23 

_ 

—  ... 

J.  M  2 

1 

MM 

.08 

.33* 

.33* 

_ 

- 



H  vs.  0  2 

3  • 

M 

.79* 

.31* 

.31* 

- 

-31* 

• 

MM  vs.  Ml.  3 

2 

m 

.67* 

.38* 

-.03 

.04 

-23 

Mt 

-.15 

-.16 

-.29 

-.28 

ML 

.46* 

-.12 

.10 

.01 

-.05 

M  vs.  L  3 

4 

M 

.82* 

.30 

-.20 

.02 

.13 

1. 

.15 

.34* 

-27 

.34* 

L 

.41* 

.51* 

.31 

.12 

.47* 

mh/mri7rk  4 

1 

MM 
MRI. 

.90* 
.33 

-.01 
-.27 

-.33 
-.23 

-.23 
-.31 

-.18 
-.35* 

RL 
MRL 

.47* 
.09* 

.28 
.04 

.23 
-.04 

.28 
.00 

RL 
MRL 

1.09* 
.42* 

.34 
.08 

.21 
-.01 

-21 
-.07 

.35 
.02 

mt  vs.  Rl.  4 

3 

m 

.63* 

.28* 

.06 

.10 

.22 

RL 

.34* 

.25 

.26* 

.29* 

RL 

.65* 

.03 

.41* 

.33* 

.28* 

M/R/I7C  4 

4 

M 

.69* 

.4  3 

.07 

'  .04 

-27 

R 

L 

C 

.50* 

.25 

.39 

.26 
.36 
.21 

.18 
.21 
.30 

.26 
.32 
.30 

R 
L 

C 

.50* 
.80* 

.25 

.48* 

-17 

.33 

.25 
.20 

.35* 
.50» 

.39 

HM/MRL/RL  3 

2 

MM 

.78* 
.38* 

.37* 
.19 

-.02 
-.27 

-.09 
.04 

RL 
MRL 

.22 
-.10 

-.03 
.03 

-.09 
-.25 

-.08 
«.14 

RL 
MRL 

.74* 
.16 

.22 
-.16 

.16 
.16 

-.38 
-.25 

.00 
-.16 

MM  vs.  RL  3 

4 

MM 

.76* 

.32* 

-.10 

-.16 

-25 

RL 

.20 

.23 

.31 

.33 

RL 

.56* 

-.09 

.21 

.33 

.15 

MH/RR/KL/RL  6 

1 

MM 

1.44* 

.16 

-.09 

-.09 

.04 

RR 

LL 

-57* 
.29 
-.13 

.26 
.28 
-.16 

.45* 
.51* 
.09 

.43* 
.47* 
-.03 

RR 

RL 
LL 

.12 

.88* 

.94* 

-.08 
.11 
-.10 

.29 
.18 
.08 

.13 
.12 
^.09 

.09 
.17 
-.08 

m/MRL/RL  6 

3 

m 

MRL 

.83* 
.48* 

.33 
.16 

.05 
-.09 

.26 
.21 

.30 
.15 

RL 
MRL 

.44 
.18 

.45* 
.51* 

.03 
-.06 

.26 
.23 

RL 

MRL 

.53* 
.25 

-.03 
-.13 

.21 
.07 

.20 
.17 

.12 
.01 

ON 

I 
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was,  in  fact,  doing  what  it  intended  to  do.    Students  who  received 

r 

mathematics  CAI  were  expected  to  do  better  on  a  test  of  that  CAI  curriculum 

than  students  who  had  not.    The  treatment  effects  on  the  mathematics  CST 

are  consistently  high,  positive  and  statistically  significant.  In 

looking  for  patterns  associated  with  the  amount  of  CAI,  the  grade  level, 

the  control  group,  and  other  complicating  factors,  two  patterns  emerge. 

Students  who  received  two  sessions  daily  of  mathematics  CAI  (MM)  reported 

treatment  effects  about  twice  the  size  of  students  who.  had  received  only 

one  session  (MRL)  in  grades  4,  5  and  6.     That  pattern  is  repeated  often 

in  Table  4-80.    A  second  pattern  involves  the  results  of  the  first 

exposure  to  a  CAI  curriculum  vs  the  results  of  later  exposures.     In  those 

cases  in  which  mathematics  CAI  students  are  compared  with  non-mxath 

sj:udents — grade  1,  grade  4  in  years  1  and  4,  and  grade  6  in  year  1 — 

treatment  effects  are  quite  high,  averaging  1.05  standard  deviations. 

Treatment  effects  after  the  first  exposure  are  lower,  averaging  .75. 

This  pattern  also  will  be  repeated  for  CSTs  in  reading  and  language. 

For  the  CTBS  computation  subtest,  the  results  for  mathematics  CAI 

students  parallel  the  CST  results,  althqugh  the  treatment  effect?  are  not 

nearly  as  high.    Omitting  the  year  1  data,  when  the  CTBS  test  was  admini- 

stered  only  two  months  after  CAI  treatments  were  initiated,  an^  also 

ignoring  for  thie  moment  students  exposed  to  the  MRL  condition,  adjusted 

means  for  students  receiving  mathematics  CAI  average  one-third  of  a 

standard  deviation  higher  than  those  for  students  withput  mathematics 

CAI./  In  grades  5  and  6  "MRL  students  have  treatment  effects  only  half. 

/  '  ^ 

the  size  of  MM  students. 

For  the  CTBS  concepts  and  applications  subtest(s),  the  results  are 
/mixed.     In  grades  1  and  2  when  the  subtest  is  read  to  students,  the 
treatment  effects  for  ma^h  CAI  studsnts  are  positive  and  statistically 
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signlflcant.    In  grades  3 


/ 


6  when  math  CAI  students  are  compared  to  CAI 


students  with  exposure  to  reading  and/or  language  and  when  students  must 
read  their  own  mathematics  problems,  the  results  are  not  consistent. 


/  / 


Reading  / 

The  reading  CAI  curriculum  was  not  used  until  grac^/^*     In  the  four 
studies  involving  the  first  exposure  of  CAI  studetits/to  the  reading 
curriculum—grade  4  in  years  1,  3  and  4  and  grade^6  in  year  1 — treatment 
effects  on  the  reading  CST  for  stu^dents  recelying  the  maximum  exposure 
average  .59  and  are  statistically  significant.     In  the  later  studies  in 
grades  5  and»6,  treatm^t  ieffects  on  the  reading  CST  average  .29  and  are 
not  signif icanjt<^ 

vocabulary  and  comprehension  subtests,  most  groups 


On^fie  CTBS 


^edeiving  reading  CAI  sliow  small  positive  treatment  effects  averaging 

about  one-quarter  of  a  standard  deviation.     Few  of  the  effects  are 

i 

statistically  significant. 

I 

! 

Language  Arts  i 

i 

The  language  CST  data  resemble  the  mathematics  CST  data  more  than 
they  do  the  reading  CST  data  in  that  there  are  statistically  significant 
treatment  effects  in  every  study.     Treatment  effects  for  first-year 
exposure  average  .74,  or  .94  without  grade  3  students.    Later  studies 
show  treatment  effects  averaging  .62.    The  content  of  the  language  CAI 
curriculum  demonstrates  more  capability  of  differentiating  between  users 
and  non-users  than  does  the  coritent  of  the  reading  curriculum. 
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^  All  of  the  statistically  significant  treatment  effects  on  the  CTBS 
language  subtests  occur  in  grades  3  and  4,     On  the  spelling  subtest, 
treatment  effects  are  variable:    mainly  positive  and  twice  significant 
in  grades  3  and  ^,  slight  and  negative  in  grades  5  and  6.  Treatment 
effects  in  language  mechanics  average  close  to  one-quarter  of  a  standard 
deviation,  and  although  the  effects  are  larger  and  sometimes  significant 
in  the  earlier  grades,  small  positive  effects  are  shown  in  grades  5  and 
6,     There  are  also  small  positive  effects  for  language  expression, 
but  it  is  interesting  that  the  largest  significant  treatment  effect 
was  obtained  by  the  group  receiving  the  newer  reading-f or-comprehension 
CAI  curriculum  in  grade  4,  year  4. 

Longitudinal  Studies 

The  longittudinal  studies  will  also  be  reviewed  with  treatment 
effects  defined  in  terms  of  standard  deviations.    Emphasis  in  this  o 
section  is  placed  on  the  CAI  curriculums  and  what  they  accomplished  in 
one*year  and  longitudinal  studies* 

Mathematics  -^^  ^ 

The  mathematics  curriculum  had  the  length  and  breadth  to  handle  all 
of  the  students  in  the  CAI.^stjjdy^byer  four-year  duration.     Only  two 
students  topped  out:     both  were  girls  who  had  been  in  the  program  for 
four  years,  receiving  20  minutes  of  mathematics  CAI  daily  before  they 
topped  out.    All  students  were  able  to  access  the  mathematics  curriculum. 
Kindergarten  students ,  l^ho  were  not  in  the  study,  visited  the  CAI  lab  in 
one  of  the  schools  on  a  regular  basis. 
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Students  at  all  grade  levels  benefited  from  the  CAI  curriculum  and 
there  were  indications  that  long~term  students  showed -continuing  gains 
over  time.     See  Table  A-81.     There  were  12  one-year  studies  in  which 
students  receiving  the  CAI  mathematics  curriculum  were  compared  with 
students  receiving  no  CAI  (two  studies),  less  math  CAI  (two  studies),  or 
other  CAI  treatments  such  as  reading  or  language  (eight  studies).  The 
average  treatment  effect  for  the  12  one-year  studies  is  .80  on  the 
curriculum  specific  test  of  mathematics,  indicating  that  students 
receiving  math  CAI  are  four-fifths  of  a  standard  deviation  higher  in 
math  performance  on  the  CST.     For  the  six  two-year  stydies  the  average 
treatment  effect  is  .91,  and  for  the  three  three-year  studies  it  is  1.23. 
Over  time,  the  mathematics  CAI  groups  increased  their  mean  distance  from 
the  non-math  CAI  group  on  the  test  of  the  CAI  curriculum^ 

On  the  standardized  test — the  CTBS — the  12  one-year  studies  showed 
an  average  treatment  effect  of  .31  of  a  standard  deviation  on  math 
computation.    That  figure  rose  to  an  average  of  .36  when  first  year 
studies  were  omitted  because  testing  occurred  only  two  months  after  the 
CAI  labs  opened.     For  the  six  two-year  studies  the  average  treatment 
effect  is  .56  and  for  the  two  three-year  studies  it  is  .72.     Over  time, 
the  mathematics  CAI  groups  increased  their  distance  from  the  non-math  CAI 
groups  on  the  CTBS  computation  subtest. 

On  the  CTBS  concepts  and  applications  subtests,  the  results  are 
less  clear.    Test  items  contain  words  to  be  read  as  well  as  mathematical 
concepts  and  applications.     In  grades  1  and  2,  concepts  and  applications 
problems  were  read  to  the  students,  and  under  those  circumstances  CAI 
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Table  4-81 

Summary  Tnble  for  One-Year  and  LonRltudlnal  Treatment  Effecta:  Mathematics 


GRADE  1 

GRADE  2 

GRADE  3 

GRADE  4 

GRADE  5 

GRADE  6 

1         2  3 
Year    Year  Year 

1         2  3 
Year    Year  Year 

1         2  3 
Year    Year  Year 

1          1  3 
Year    Year  Year 

1          2  3 
Year    Yea^  Year 

1         2  3 
Year    Year  Year 

YEAR  1 
CST 

COMPirrATION 

CONCEPTS 

APPLICATION 

.08 

.35* 

.33* 

.90* 
-.01 
-.33 
-.23 

1 .44* 

.16 
-.09 
-.09 

YEAR  2 
CST 

COMPUTATION 

CONCEPTS 

APPLICATION 

1.16* 
.12 
.37* 

.67* 
.38 
-.03 
.04 

.78*  I.IO* 

.37*      .63*  ^ 
-.02  .21 
-.09  .07 

YEAR    3  ,  ^ 
CST 

COMPUTATION 

CONCEPTS 

APPLICATION 

.79*  .66* 
.51*  .67* 
.31*  .50 

.63*  .99*.. 
.28*  .59* 
.06  .21 
.10  .36 

.83*    .65*    .97 • 
.33      .28  .64* 
.05    -.03  .28 
.26      .20  .58 

YEAR  U 
CST 

COMPUTATION 

CONCEPTS 

APPLICATION 

.82*      .91*  1.54* 
.30  .61 
-.20  .44 
.02  .19 

.69* 
.43 
.07 
.04 

.76*      1.18*  1.17* 
.52*        .57*  .79* 
-.10       -.21  -.10 
-.16        -.20  -.05 

%  <.05 
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treatment  effects  in  the  one-year  studies  average  one-third  of  a  standard 
deviation  and  are  statistically  significant.    The  one  two-year  study  showed 
a  treatment  effect* of  .50.     In  grades  3-6  students  read  their  own  test 
questions,  and  since  math  CAI  students  were  being  compared  with  students 
receiving  reading  and/or  language  CAI,  the  results  may  have  been  confounded. 
No  statistically  significant  treatment  effects  for  either  concepts  or 
applications  occur  in  grades  3-6.     Table  4-82  and  Figure  4-2  summarize  the 
final  estimates  of  treatment  effects  in  mathematics.     Overall,  the  mathematics 
strands  curriculum  performed  very  well.     It  adapted  to  students  of  all 
ability  levels '^and  provided  effective  drill  and  practice  in  mathematics 
computation.     Its  effectiveness  was  demonstrated  both  in  one-year  studies 
and  over  two-  and  three-year  periods. 

Table  4-82 

Summary  of  the  1-Year,  2-Year,  and  3-Year 
Studies  of  Mathematics  CAI 


r  Number  of 

Studies  Kean  Standardized         Performance  Level 

Averaged  Treatment  Effects         in  Percentiles 


Tests 

lYR 

2YR 

3YR 

lYR 

2YR 

3YR 

lYR 

2y.R 

3YR 

Math  CST 

12 

6 

3 

.  80** 

.91** 

1.23** 

79 

82 

89 

CTBS  Computation 

9 

0 

2 

.  36** 

.56** 

.72** 

64 

71 

76 

CTBS  Concepts 

7 

5 

2 

-.02 

.12 

.09 

49 

55 

54 

CTBS  Applications 

7 

5 

2 

.03 

.12  , 

.26 

51 

55 

60 

CTBS  Concepts  & 

2 

1 

0 

.34** 

.50 

63 

69 

Applications-*- 

\ 

**    p  <  .01. 

^In  grades  1  and  2  Concepts  and  Applications  is  a  single  subtest. 
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Af  ter 
1 

Year 


After 
2 

.  Years 


After 
3 

Years 


CST:  MATH 

(Grades  1-6) 


CTBS;  COMPUTATION 
(Grades  1-6) 


CTBS:    CONCEPTS  &  APPLICATIONS 
(Grades  1-2) 


CTBS;  APPLICATIONS 
(Grades  3-6) 

CTBS;  CONCEPTS 


Figure  4-2      Mathematics  Treatment  Effects  Over  3  Years, 


Reading  and  Language  , 

The  CAI  reading  curriculum  used  in  this  study  was  developed  for 
students  in  grades  3-6,  with  an  addition  of  basic  sentences  which  purported 
to  reach  students  at  grade  level  2.5.     For  the  students  in  our  study, 
those  estimates  were  misleading.    At  grade  4  many  students  assigned  to 
the  reading/language  (RL)  CAI  treatment  were  transferred  to  the  MM  group 
after  six  to  eight  weeks.     CAI  coordinators  kept  students  in  the  assigned 
curriculums  for  four  to  six  weeks  after  the  rapid  motion  phase  of  the  CAI 
curriculum  had  placed  students  at  their  own  ability  levels.     When  a 
student  had  made  absolutely  no  progress  after  four  to  six  weeks  and  the 


333 
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curriculum  wias  deemed  unsuitable  by  the  CAI  coordinator,  the  student's 
curriculum  was  changed.     In  one  of  the  Title  I  schools  the  most  frequent 
reason  for  the  change  was  that  the  student  was  a  non-reader*     In  the 
other  Title  I  school  the  most  frequent  reason  was  that  the  student  was 
Spanish-speaking.     It  is  clearly  a  drawback  when  accessibility  to  a 
reading  drill-and-practice  CAI  curriculum  is  dependent  on  the  ability  to 
read  relatively  well.     On  the  other  hand,  at  grade  6  the  reading  CAI 
curriculum  sometimes  proved  to  be  too  easy.     A  few  students  topped  out  of 
the  reading  curriculum  during  the  rapid  motion  phase.    A  few  students 
assigned  to  two  sessions  of  reading  daily  topped  out  within  a  few  months, 
although  generally  speaking  progress  was  -very  slow  in  the  reading  curriculum. 

The  language  curriculum  was  designed  for  students  in  grades  3-6,  and 
because  its  vocabulary  was  simpler  than  that  in  the  reading  curriculum  it 
caused  fewer  problems  of  accessibility  in  the  early  grades.     On  the  other 
hand,  it  caused  more  frequent  problems  in  grade  6  with  students  topping 
out.     Relatively  large  numbers  of  students  who  were  assigned  to  two  sessions 
of  language  CAI  daily  in  the  latter  half  of  sixth  grade  topped  out  of  the 
curriculum  in  the  rapid  motion  phase,  and  others  topped  out  within  a  few 
months.    Progress  in  the  language  curriculum  was  more  rapid  than  progress 
in  reading.     For  students  interested  in  achieving,  the  more  rapid  progress 
was  pleasing. 

The  length  and  breadth  of  the  reading  and  language  CAI  curriculums 
did  not  purport  to  be  as  great  as  the  mathematics  CAI  curriculum. 
Perhaps  this  is  less  of  a  problem  for  the  use  of  the  curriculums  in 
elementary  schools  than  it  was  for  the  evaluation  of  the  curriculums  in 
this  study.     Schools  after  all  would  not  generally  assign  students 
randomly  to  a  CAI  curriculum.     However,  there  are  limitations  to  the  use 


of  these  specific  reading  and  language  -curficulums  even  though  they  were 
the  broadest  available  when  the  study  started.     Perhaps  with  newer  technology 
a  reading/language  CAI  curriculum  can  be  built  which  will  have  the  broad 
applicability  that  the  mathematics  curriculum  enjoys. 

Table  A-83  summarizes  CAI  treatment  effects  in  the  one-,  two-  and 
three-year  studies  of  reading  and  language.     The  reading  results  will  be 
discussed  first.     Since  the  CAI  reading  curriculum  was  used  only  in  grades 

3 

4-6,  only  seven  one-year  studies  are  applicable.     Two  of  those"  studies 
occurred  in  the  first  year  when  CAI  labs  did  not  open  until  late  January 
or  February,  yet  those  treatment  effects  are  among  the  highest  found. 

For  the  six  one-year  studies  in  which  students  assigned  to  both 
reading  and  language  CAI  were  compared  to  MM  students,  the  mean  treatment 
effects  for  the  CST  and  the  vocabulary  and  comprehension  subtests  of  the 
CTBS  are  .33,   .2A  and  .21.     For  students  assigned  only  to  reading  CAI  (grade 
6,  year  1  and  grade  A,  year  A)  those  figures  are  .53,  .26  and  .31.  The 
means  of  the  8  treatment  groups  receiving  reading  CAI  are  .38  on  the  CST, 
.25  on  CTBS  Vocabulary  and  .23  on  CTBS  Comprehension.     For  the  three^two-year 
studies  dealing  exclusively  with  fourth  to  sixth  grade,  the  mean  treatment 
effects  are  .52,  .17  and  -.01.     The  solitary  three-year  study  shows  treatment 
effects  of  .A2,  .58  and  -.2A.     Those  mean  treatment  effects  are  shown  in 
Table  A-8A  and  Figure  A-3. 

On  the  basis  of  the  single  three-year  study,  one  might  reach  the 
tentative  conclusion  that  over  the  three  years  the  RL  group  improved  its 
vocabulary  skills  and  lost  comprehension  skills.     The  evidence  is  certainly 
not  strong.    Whether  a  consistent  pattern  fails  to  emerge  because  of  some 
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Table  4-83 


Suasury  Table  for  One-"Year  and  Longitudinal  TreaCMenc  Effecci:    Reading  and  Language 


CRAPE  3 


GRADE  4j 


GRADE  3 


GRADE  6 


One  Longitudinal 
Year  Two  Three 
Study    Yean  Yean 


One  Longitudinal 
Year  Two  Three 
Study     Yean  Years 


One  Longitudinal 
Year  Two  Three 
Study     Yean  Yeara 


One  Longitudinal 
Year  Two  Three 
Study      Years  Years 


YEAR  I 

READINC-^ 
CST 

VOCABULARY 
COMPREHENSION 
LANGUAGE 
CST 

SPELLING 

MECHANICS 

EXPRESSION 


RL  vs.  MM 


.47* 

.28 

.23 

1.09  • 
.34 
.21 
.21 


ONE-YEAR  STUDY  vs.  MM 
RK         RL  LL 


.57* 
.26 
.4  5* 

.12 
-.08 
.29 
.13 


.29 
.28 
.51* 

.88* 
.11 
.18 
.12 


-.13 
-.16 
.09 

.94* 
-.10 

.08 
-.09 


YEAR  2 

READING 
CST 

VOCABULARY 
COMPREHENSION 
LANGUAGE 
CST 

SPELLING 

MECHANICS 

EXPRESSION 


ML  vs.  MM 


-.15 
-.16 
-.29 


.46* 
-.12 
.10 
.01 


RL  va.  MM 


.22  .50* 

.03  *  .08 

.09  -.07 

.74*  .74* 

.22  .46* 

.16  .34 

.38  -.12 


YEAR  3 

READING 
CST 

VOCABULARY 
COMPREHENSION 
LANGUAGE 
CST 

SPELLING 

MECHANICS 

EXPRESSION 


RL  vs. 


MM^ 


RL  vs.  MM 


.34*  ^16 

.25  .04 

.26*  .02 

.65*  .59* 
.03  -.22 

.41*  .34 

.31*  .15 


.44 

.45* 

.03 

.53* 
.03 
.21 
.20 


.32  .42 

.28  .58 

.24  -.24 

.57*  .76* 

.06  .22 

.18  .09 

.04  .22 


YEAR  4 

READING 
CST 

VOCABULARY 
COMPREHENSION 
LANGUAGE 
CST 

SPELLING 

MECHANICS 

EXPRESSION 


p  <.05 


L  vs.  M 


•  15 
.34* 

.27 


.41* 
.51* 
.31 
.12 


ONE-YEAR  STUDY  vs.  M 
R         L  C 


.50*  .25  .39 

.26  .36  .21 

.18  .21  .30 

.50*  .80*  - 

.25  .48  .17 

.33  .44  .33 

.25  .20  .45 


RL  v8 


.20 

.73* 

.  31 

.23 

.14 

-.04 

.31 

.27 

.24 

1.14  • 

.71* 

-.09 

.02 

.06 

.21 

.22 

.40 

.33 

.13v 

.24 

Th«  lait  column  of  figures  In  these  two  cells  are  based 
on  regressions  from  Grade  3,  where  the  trentment  was 
ML  rather  than  RL. 
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Table  4-84 

Summary  of  the  1-Year,  2-Year,  and  3-Year 
Studies  of  Reading  CM 


Tests 

Reading  CST 
CTBS  Vocabulary 
CTBS  Comprehension 


Number  of 

Studies 

Averaged 

lYR    2YR  3YR 


8 

3 

1 

8 

3 

1 

8 

3 

1 

Mean  Standardized 
Treatment  Effects 

lYR  2YR  3YR 

.38**  .52**  .42 

.25**  .17  .58 

.23**  -.01  -^.24 


Performance  Level 
in  Percentiles 

lYR  2YR  3YR 

65  70  66 

60  57  72 

59  50  ^1 


**    p  <  .01. 
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quality  of  the  curriculum/  problems  associated  with  the  bottoming-out  and 
topping-out  phenomena,  or  some  other  factor,  one  cannot  tell.     One  can 
compare  the  treatmentef f ectd  of  students  receiving  reading  CAI  with  the 
effects  associated  with  other  verbally  oriented  CAI  curriculums  in  grade 
4,  year  4.     (See  Table  4-83.)     In  that  one-year  stvdy  students  assigned 
to  language  CAI  or  the  newer  reading-for-comprehension  curriculum  also 
show  positive  (but  not  significant)  treatment  effects  on  reading  dependent 
variables.     Although  the  reading  group's  treatment  effects  are  best  on 
the  reading  CST,  the  laiiguage  group  is  best  on  vocabulary,  and  the 
reading-for-comprehension  group  best  on  comprehension. 

Turning  to  the  CAI  language  treatment  effects,  we  see  in  Table  4-83 
that  there  are  nine  one-year  studies.    As  was  the  case  with  the  reading 
results,  the  CST  treatment  effects  in  the  first  year  of  the  study  are 
the  largest  obtained  in  the  four  years  even  though  students  had  received 
only  four  months  of  CAI  in  that  school  year.     Averaging  the  treatment 
effects  of  students  who  received  language  CAI  in  grades  3-6,  we  note  that 
all  effects  in  the  one-year  studies  involving  the  CSTs  are  statistically 
significant.     That  is  different  from  the  reading  CST  results  and  shows  a 
greater  differentiation  between  the  MM  and  RL  groups  when  tested  ou  the 
content  of  the  language  curricultim  than  when  tested  on  the  content  of  tu<» 
reading  curriculum.     Mean  CAI  treatment  effects  for  one-,  two-  and  three-year 
studies  of  the  language  curriculum  are  presented  in  Table  4-85  and  Figure  4-4 
Means  were  averaged  over  10  sets  of  treatment  effects  in  the  nine  one-year 
studies,  four  sets  of  two-year  studies  and  two  sets  of  three-year  studies. 
Although  all  the  treatment  effects  are  positive,  they  fail  to  show  the 
pattern  of  increasing  gains  demonstrated  by  the  mathematics  CAI  curriculum. 


Table  4-85 


Summary  of  the  1-Year,  2-Year  and  3-Year 
Studies  of  Language  CAI 


Number  of 

Studies 

Averaged 

Mean  Standardized 
Treatment  Effects 

Performance  Level 
in  Percentiles 

Tests 

lYR 

2YR 

3YR  . 

lYR 

2YR 

3YR 

lYR 

2YR 

3YR 

Language  CST 

10 

4 

2 

.71** 

.76** 

.73** 

76 

78 

77 

CTBS  Spelling 

10 

4 

.  2 

.14* 

.05 

.14 

56 

52 

56 

CTBS  Mechanics 

10 

4 

2 

.22** 

.27* 

.25 

59 

61 

60 

CTBS  Expression 

10 

-4 

2 

.11 

.05 

.23 

54 

52 

59 

*  p  <  .05. 
**    p  <  .01. 


After 
1 

Year 


After 
2 

Years 


After 

3 

Years 


CST:  LANGUAGE 


 .CTBS: 

^  —  CTBS: 
'CTBS: 


MECHANICS 

EXPRESSION 

SPELLING 


Figure  4-4    Language  Treatment  Effects  over  3  Years. 
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•When  the  treatment  effects  in  Table  4-85  are  compared  with  the 
one-year  effects  for  students  receiving  reading  CAI  or  reading-for- 
comprehension  CAI  in  grade  4,  year  4  (Table  4-83),  the  CTBS  results  for 
those  students  in  one  year  equal  or  surpass  the  treatment  effects  averaged 
for  CAI  language  across  one,  two  or  three  years •    The  reading-for-compre- 
hension  CAI  group  is  significantly  better  in  language  expression,  and  the 
reading  CAI  group  put  in  a  good,  solid  performance.     Clearly  there  is 
overlap  in  the  ability  of  the  three  verbally  oriented  CAI  curriculums  to 
produce  results  on  CSTs  and  CTBS  subtests  in  both  reading  and  language. 
Although  each  of  the  three  verbally  oriented  CAI  curriculums  has  demon- 
strated some  ability  to  help  students  perform  better,  they  have  yet  to 
demonstrate  the  strong,  overall,  consistent  effects  achieved  by  the 
mathematics  curriculum* 

CAI  vs  No  CAI 

*         ♦  -' 

The  within-CAI  analyses  reported  in  the  first  two  sections  of  this 
chapter  give  the  best  estimates  of  treatment  effects  that  we  can  obtain 
from  the  data.     The  randomized  assignment  of  students  to  CAI  curriculums-- 
everi  when  implemented  with  less  than  absolute  success^ — assured  the  study 
of  its  best  control  groups.     Two  other  control  groups  were  used,  however. 
Cohort  controls  were  students  who  were  in  CAI  schools  one  year  earlier 
or  later  than  CAI  students  and  who  were  not  given  access  to  the  CAI  labs. 
Comparison  students  were  in  the  same  area  of  Los  Angeles  as  CAI  students, 
in  two  elementary  schools  without  CAI  facilities.    Before  we  report  on  CAI 
effects  when  CAI  students  are  contrasted  with  cohort  controls  or  comparison 
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students,  we  will  explain  the  reservations  we  have  about  these  regression 
analyses* 

With  regard  to  the  cohort  control  analyses,  by  their  very  nature 
they  cut  across  testirig  years.     Although  the  school  and  teacher  variables 
are  better  controlled  than  those  for  comparison  schools,  effects  specific 
to  the  year  of  testing  cannot  be  controlled.    Earlier  or  later  testing 
dates,  the  amount  of  time  between  pretesting  and  posttesting,  the  loss  of 
time  due  to  flu  epidemics  and  other  conditions  specific  to  a  school  year 
may  affect  the  findings.     During  posttesting  at  the  end  of  year  3, 
stude'nts  found  the  body  of  a  young  man  shot  to  death  in  a  suspected 
gang-related  incident  near  the  school  playground  of  one  of  the  CAI  schools. 
For  one  week  in  that  testing  period  conditions  at  the  school  were  unlike 
conditions  obtaining  at  any  other  period  of  the  study. 

Analyses  involving  CAI  schools  vs  comparison  schools  suffer  because 
of  school  differences.     In  three  years  of  classroom  observations,  comparison 
school  students  were  found  to  spend  more  time  on  task  overall,  comparison 
teachers  spent  almost  twice  as  much  time  teaching,  and  comparison  teachers 
were  rated  higher  by  observers  on  teaching  effectiveness.     The  smaller  of 
the  two  comparison  schools  was  a  Title  I  school;  more  than  two-thirds  of 
the  CAI  students  were  from  Title  I  schools. 

With  those  reservations  recorded,  the  treatment  effects  will  be 
examined. 

Mathematics 

Table  4-86  presents  an  overview  of  the  CAI  treatment  effects  in 
mathematics  for  CAI  students  vs  cohort  controls  and  comparison-school 

students. 

'J 
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Table  4-86 


Treatment  Effects  for  CAI  Students  vs  Cohort 
Controls  and  Comparison  School  Students:  Mathematics 


Grades 


CAI  vs  Cohort  Controls 


CST 

COMPUTATION 

CONCEPTS 

APPLICATIONS 


.80* 
.51* 
.46* 
.38* 


CAI  vs  Comparisons 

.76* 
.15 
.60* 
.87* 


Grades 


> 


CAI  vs  Cohort  Controls 


CST 

COMPUTATION 

CONCEPTS 

APPLICATIONS 


m  RL 

1.26*  .46* 

.70*  .08 

.18  .11 

.28  .22 


CAI  vs  Comparisons 
MM  RL 


1.02* 
.45* 

-.03 
.29 


.11 
-.03 
-.02 

.21 


Grades 


CAI  vs  Cohort  Controls 

MM  RL 


CST 

COMPUTATION 

CONCEPTS 

APPLICATIONS 


1.11*  .57* 
.08  -.07 
.20  .06 
.07  -.26 


CAI  vs  Comparisons 
MM  RL 


.28  -.20 

-.19  -.32 

.09  .14 

.04  -.11 


<  .05 
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In  the  cohort-control  data  we  see  a  trend  for  the  statistically 
significant  effects  to  occur  at  the  lower  grade  levels.     It  is  difficult 
to  know  whether  that  is  a  meaningful  observation  or  an  artifact.  The 
grade  4-6  CAI .group  had  many  difficulties:    randomization  problems, 
differential  attrition,  loss  of  students  to  the  LAUSD  desegregation  plan. 
Forced  to  estimate  a  treatment  effect  for  CAI  students  vs  cohort  controls, 
one  might  aj^erage  the  effects  for  mathematics  CAI  students  to  come  up 
with  1.06  for  the  mathematics  CST,  .43  for  CTBS  math  computation,  .28  for 
concepts  ahd  .24  for  applications.     Those  estimates  are  slightly  lower, 
overall,  than  the  three-year,  within-CAI  estimates  in  Table  4-82. 

The  comparison-school  data  repeat  the  pattern  of  more  significant 
treatment  effects  in  the  lower  grades  than  in  the  upper. 

Reading  and  Language  \ 

Table  4-87  presents  the  treatment  effects  for  CAI  students  vs  cohort 

I 
1 

controls  and  comparison  students  on  reading  and  language  tests.     One  of 

1 

the  interesting  things  to  notice  is  the  differential  pe|rf ormance  of  the 
RL  group  on  reading  CSTs  as  compared  to  language  CSTs  in  these  analyses. 
In  the/  cohort  control  studies  the  RL  group  performs  onl'y  half  as  well  on 
readl/ng  CSTs  as  they  do  on  language  CSTs.     The  contrast  is  even  more  • 
striking  in  the  comparison  school' data.     The  language  arts  CAI  curriculum 
teaches  students  something  which  helps  to  differentiate  thein  from  students 
without  that  curriculum  when  they  are  tested  on  the  CSTs.     That  cannot  be 
said'  as  strongly  for  the  reading  curriculum. ' 

Estimating  overall  treatment  effects  for  CAI  vs  cohort  controls  with 
no  CAI,  we  find  the  following:  .44  for  the  reading  CST,  .22  for  the  CTBS 

^  399 
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Table  4-87 

Treatment  Effects  for  CAI  Students  vs  Cohort  Controls  and 
Comparison  School  Students:    Reading  and  Language 


Grades  2345  2345 

I  >       I  > 

CAI  vs  Cohort  Controls  CAI  vs  Comparisons 


RL 

m 

RL 

m 

.56* 

.23 

.11 

-.02 

Lloo  VULiAJt>UJLiAKl 

.41* 

.34* 

-.04 

-.09 

COMPREHENSION 

.20 

.17 

-.27 

-.21 

LANGUAGE  CST 

1.06* 

.45* 

.46* 

.11 

CTBS  SPELLING 

.20 

.20 

-.13 

-.05 

MECHANICS 

.11 

.18 

.12 

.15 

EXPRESSION 

.22 

.28 

.08 

-.03 

Grades           ^  4 

5 

6 

4  

6 

 v> 

 ^ 

CAI  vs  Cohort 

Controls 

CAI  vs  Comparisons 

Mi 

m 

M. 

MM 

READING  CST 

.32 

-.15 

.26 

-.33 

CTBS  VOCABULARY 

.03 

-.33 

.02 

.14 

COMPREHENSION 

-.05 

.11 

.44 

-.01 

LANGUAGE  CST 

.66* 

-.05 

.70* 

-.21 

CTBS  SPELLING 

.11 

-.11 

.08 

-.15 

MECHANICS 

.32 

-.12 

.27 

-.21 

EXPRESSION 

.12 

-.12 

.12 

-.11 

p  <  .05 


•i 

'lUQ 
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vocabulary  subtest  and  .07  for  comprehension.     In  language  we  find.  .86 
for  the  language  CST,  .15  for  CTBS  spelling,   .21  for  language  mechanics 
and  .17  for  language  expression.     The  reading  results  may  be  compared 
with  the  results  of  the  within-CAI  three-year  analyses  in  Table  4-84. 
They  are  not  consistent.     Language  results  for  GAI  vs  cohort  controls  are 
more  consistent  with  the  within-CAI  results  in  Table  4-85. 

For  the  comparison  school  data,  we  note  that  most  of  the  results  for 
the  RL  group  are  positive,  while  most  for  the  MM  group  are  negative.  On 
the  average,  the  performance  of  the  RL  group  is  at  least  as  good  as  the 
performance  of  the  comparison  school  students. 
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Chapter  VI  ^ 
SUICIARY  AND  CONCLUSIONS  y/ 

In  part  4  of  the  Final  Report  we  have  been  examining  the  effectiveness 
of  drill-and-practice  CAI  curriculums  at  improving  student  performance  in 
mathematics,  reading  and  language  arts.     The  statistical  model  underlying 
the  bisic  analysis  was  reported  in  Chapter  II.    We  learned  that  the  basic 
regressions  used  sex,  ethnicity,  classrooms,  pretests  and  CAI  treatments 
.to  predict  student  outcomes  both  on  standardized  tests — the  CTBS — ^and  on 
curriculum-specific  tests — CSTs — developed  from  the  CAI  curriculums.  The 
CSTs  were  used  to  measure  whether  or  not  each  of  the  CAI  curriculums  was 
successful  in  improving  the  performance  of  students  on  the  material  in 
which  it  was  drilling  them.     The  CTBS  was  used  to  measure  the  CAI  cur- 
riculum's ability  to  change  students'  performance  on  standardized  tests. 

In  Chapter  III  we  saw  the  results  of  the  within-CAI  regression 
analyses  done  for  the  12  one-year  studies.    We  noted  that  first-  and 
second-grade  students  were  able  to  use  the  mathematics  CAI  curriculum 
effectively,  that  students  in  grade  3  used  the  mathematics  and  language 
curriculums  effectively',  and  that  students  in  grades  4^6  were  able,  for 
the  most  part,  to  use  all  three  of  the  CAI  curriculums:  mathematics, 
language  arts  and  reading.     Some  statistically  significant  treatment 
effects  were  noted  at  each  grade  level  both  on  the  curriculum-specific 
tests  and  the  standardized  tests. 

Two  findings  from  the  one-year  studies  may  be  easily  overlooked  in 
the  mass  of  data  which  were  presented.     One  finding  involved  the  second- 
grade  use  of  fixed  strands  in  the  mathematics  CAI  curriculum.  Ordinarily 

the  mode  of  presentation  in  the  mathematics  strands  cuifriculum  was  variable 

\ 
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strands,  where  a  problem  In  horizontal  addition  might  be  followed  by  one 
in  vertical  subtraction  or  number  concepts.     In  fixed  strands,  problems 

received  by  the  student  were  limited  for  the  session  to  one  strand 

\ 

selected  by  the  teacher  or  CAI  coordinator.     In  year  3,  students  in 
grade  2  were  randomly  selected  within  classrooms  to  receive  part  of  their 
mathematics  drill-and-practice  via  fixed  strands.     Although  there  were  no 
statistically  significant  differences  between  the  two  groups,  treatment 
effects  consistently  favored  the  fixed-strand  group  on  the  mathematics 
CST  total,  CTBS  computation  and,  most  strongly,  concepts  and  applications 
The  findings  suggest  that  the  treatment  effects  for  the  mathematics 
strands  curriculum* might  have  been  higher,  had  the  study  routinely 
allowed  the  provision  of  some  CAI  time  in  the  fixed-strand  mode.  One 
simple  procedure  used  it  School  1  was  to  assign  the  student  to  f\xed 
strands  in  that  strand  with  the  lowest  placement  score  on  the  student's 
CAI  record.     Fixed  strands  were  used  for  only  patt  of  the  students*  CAI 
time;  the  majority  of  the  time  was  spent  in  variable  strands.  The 
use  of  fixed  strands  as  described  was  somewhat  akin  to  a  tutorial  session 
in  the  student's  area  of  weakness.     The  combination  of  fixed  and  variable 
strands  was  not  significantly  better  than  variable  strands  alone,  but 
it  was  consistently  slightly  better  and  was  approved  by  CAI  coordinators 
even  though  it  was  more  work  for  them.' 

The  second  finding  that  could  be  overlooked  in  the  one-year  studies 
was  the  effectiveness  rating  of  the  newer  reading-f or-comprehension 
curriculum  used  in  grade  4,  year  4.     The  reading-f or-comprehensioti 

curriculum  differed  from  the  older  reading  curriculum  in  one  major  way: 

t 

\ 
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in  every  CAI  session,  the  reading-f or-comprehension  curriculum  presented 
a  paragraph  followed  by  five  questions  very  similar  in  format  to  questions 
in  reading  comprehension  subtests.    When  the  performance  of  students 
assigned  to  the  reading-f or-comprehension  curriculum  (C),  the  reading 
curriculum  (R)  and  the  language  curriculum  (L)  was  compared  to  the 
performance  of  students  assigned  to  mathematics  CAI,  the  results  show 
positive  treatment  effects  for  all  three  groups  on  reading  and  language 
tests.    Group  C  put  in  the  best  performance  on  the  reading  comprehension 
subtest  and  a  significantly  better  performance  in  language  expression. 
Although  the  newer  comprehension  curriculum  was  only  studied  for  one  year 
without  replication  or  longitudinal  data,  the  one-year  data  suggests 
increased  potential  in  the  reading  curriculum  area.     The  paragraph 
questions  also  appealed  to  students,  teachers  and  coordinators  as 
ah  improvement  over  the  straight  reading  curriculum. 

In  Chapter       we  examined  the  three  longitudinal  studies.    We  saw 
that  the  mathematiqs  CAI  curriculum  was  effective  in  improving  students' 
performance  in  grades  1-3  both  on  CSTs  and  standardized  tests.  Students 
in  grades  2-5  started  CAI  with  two  levels  of  mathematics  and  became 


increasingly  differentiated  into  an  IM 


group  and  an  RL  group.  Their 


results  in  mathematics  paralleled  the  ::esults  of  the  earlier  CAI  cohort. 
The  results  in  reading  and  language  were  generally  positive  and  ocassionally 
significant.     The  students  in  grades  4|-6  experienced  several  problems 
which  may  have  hurt  their  longitudinal  study.     The  random  assignment  of 
students  to  curriculums  was  compromised  when  non-readers  and  some  Hispanic 
students  could  not  function  in  the  reading  and  language  curriculums. 
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Differential  attrition  rates  over  three  years  found  brighter  students  leaving 
the  Mtl  and  MRL  groups  and  slower  students  leaving  the  RL  groups.  The 
fourth  to  sixth  grade  study  was  also  hardest  hit  by  the  LAUSD  desegregation 
plan  when  numbers  of  students  in  grades  5  and  6  moved  out  of  CAI  schools.  , 
Nevertheless,  treatment  effects  in  mathematics  followed  a^  pattern  similar 
to  the  earlier  longitudinal  studies.     Results  in  reading  were  not  impressive. 
Language  results  were  generally  positive  for  both  CSTs  and  CTBSs  and  were 
consistently  significant  for  the  language  CSTs. 

In  Chapter  V  we  estimated  treatment  effects  for  the  three  CAI 
^     curriculums  and  evaluated  them  in  terms  of  their  usefulness  to  the 
population  of  students  in  the  study.     The  mathematics  curriculu^  was 
found  to  have  the  widest  applicability ,  capable  of  being  used  with 
students  from  kindergarten  through  sixth  grade.     Treatment  effects  for 
mathematics  CAI  students  were  averaged  across  grade  levels  to  obtain  a  ^ 
mean  effect  in  one-,  two-  and  three-year  studies.     For  the  CST  data,  mean 
treatment  effects  for  the  mathematics  CAI  students  rose  from  .80  for 
one-year  studies,  to  .91  for  two-year  studies,  to  1.23  for  three-year 
studies.     CTBS  computation  data  reflected  the  same  pattern  with  mean 

treatment  effects  rising  from  .36  for  one-year  studies,  to  .56  for 

f 

two-year  studies,  to  .72  for  three-year  studies.     Only  the  mathematics 
curriculum  provided  this  pattern  of  increasing  differentiation  between 
groups  over  time.     One  is  compelled  to  believe  in  the  mathematics  curriculum 
as  a  valuable  resource  in  the  area  of  mathematics  training  for  elementary- 
school  students. 

The  reading  and  language  CAI  ciirriculums  were  applicable  at  fewer 
grade  levels  and  for  fewer  students  within  the  appropriate  grade  levels. 

•J 
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One  of  the  biggest  drawbacks  in  the  reading  drill-and-practice  curriculum 
is  the  pre-admission  requirement  of  a  third-grade  reading  ability.  With 
newer  technology  a  broader  approach  to  reading  may  be  possible  and  would 
be  welcomed •     As  the  reading  and  language  curriculums  now  stand,  Hispanic s 
and  non-readers  in  this  study  had  difficulty  making  any  progress,  especially 
in  the  reading  curriculum.     Assignment  of  Spanish-speaking  children  and 
non-readers  to  either  curriculum  required  large  amounts  of  tutoring  and 
translating  time  on  the  part  of  personnel  in  the  CAI  lab.     Eventually  the 
curriculums  were  judged  unsuitable  for  some  of  those  students.  Other 
students  readily  topped  out  of  the  reading  and,  more  frequently,  the 
language  CAI  curriculums  in  grade  6. 

On  Che  CSTs  the  treatment  effects  for  RL  students  over  one,  two  and 
three  years  were  .44,   .52  and  .42  for  the  reading  CST  and  .70,   .76  and 
l73  for  the  language  CST.     Treatment  effects  in  reading  were  significant 
only  for  the  first  year  of  exposure  to  the  CAI  reading  curriculum.  All 
treatment  effects  on  the  language  CST  were  statistically  significant. 
There  appears  to  be  more  "curriculum"  in  the  language  arts  software  than 
in  the  reading  software.     On  the  CTBS  data,  small  positive  effects — seldom 
significant  after  grade  4 — appear  to  characterize  the  treatment  effects 
for  the  RL  groups  in  reading  and  language.     Perhaps  no  strong  patterns 
emerge  because  the  curriculum  was  not  broad  enough  and  the  students 
received  only  half  as  much  of  each  curriculum  as  the  Mil  students  received 
in  mathematics. 

When  CAI  students  were  compared  to  their  cohort  controls  in  the  same 
schools  one  year  earlier  or  later,  estimated  treatment  effects  in  three-year 
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studies  were  very  similar  to  the  three-year  within-CAI  treatment  effects  in 
mathematics  and  language  arts.    leading  effects  were  less  consistent. 

In  summary,  we  can  say  that  each  of  the  CAI  curriculums  proved 
its  effectiveness,  although  some  curriculums  performed  better  than 
'others.    The  mathematics  curriculum  showed  strong  promise,  in  longitudinal 
studies.    The  reading  and  language  CAI  curriculums  have  less  breadth 
but  are  both  capable  of  helping  students  to  improve.  Computer-assisted 
instruction  as  defined  in  this  study  is  a  powerful  tool  for  increasing 
student^'  skills  in  mathematics,  reading  and  language  arts. 
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PART  5* 


COMPUTER-ASSISTED  INSTRUCTION  AND  COMPENSATORY  EDUCATION: 
THE  ETS/LAUSD  STUDY 


Longitudinal  Patterns  of  Student  Attitudes 
IN  A  Computer-Assisted  Instruction  Curriculum 


Philip  A.  Griswold^  Ed.D, 
Eastern  [^ntana  College 


*This  appeared  as  Part  5  of  M.  Ragosta,  P.  W.  Holland,  and  D.  T.  Jamison 
Computer-Assisted  Instruction  and  Compensatory  Education:  The  ETS/LAUSD  Study. 
Final  Report  to  the  U.  S.  National  Institute  of  Education,  April  1982. 
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■        •  INTRODUCTION  • 

The  Coleman  report  (Coleman  et  ^d.,  1966)  made  available  informa- 
tion' concerning  the  relationship  between  achievement  and  student  attitudes. 
Subsequent  studies  also  revealed  correlations  between  achievement  and 
attitudes  such  as  attitudes  toward  school  (Bloom,  1976;  Jackson,  1968), 
attitudes  toward  reading  (Engin,  Wallbrown,  &  Brown,  1976;  Roettger, 
Szymczuk,  &  Millard,  1980)  and' mathematics  (Aiken,  1976)  ,  academic  self- 
concept  (Scheirer  &  Kraut,  1979),  and  academic  locus  of  control  (Crandall, 
Katkovsky,  &  Crandall,  1965;  Lef court,  1976).     These  researchers  were 
interested  in  the  influence  attitudes  had  on  academi':  achievement. 

In  addition  to  their  role  as  a  mediating  variable  for  achievement, 
positive  attitudes  should  be  an  Important  outcome  of  schooling.  Individuals 
develop-much  of  their  attitudes  toward  self  during  the  period  in  which 
they  attend  school  (Shaffer,  1979).  Bloom  (1976)  has  reported  some 
evidence  and  speculated  that,  positive  attitudes  in  school  have  major 
effects  on  the  individual's  later-  mental  health.    McMillaiv  (1980)  has 
likened  student  attitude  development  to  a  basic  skill,  because  of  its 
prerequisite  nature  for  effective  cognitive  learning  and  positive  mental 
health.  / 

Encouragement  of  positive  attitudes  toward  achievement  has  been  an 
important  objective  of  compensatory  education    (Passow,  1974).  Attitudes 
should  be  an  integral  part  of  the  goals  of  the  schooling  process. 
Therefore,  effectiveness  of  educational  intervention  should  include  an 
evaluation  of  its  impact  on  student  attitudes,  independent  of  achievement. 

The  effectiveness  of  computer-assisted  instruction  (CAI)  in  demon- 
strating affective  gains  is  not  as  clear  as  the  evidence  for  cognitive 


gains.     In  an  evaluation  of  the  PLATO  system,  Swinton,  Amarel,  and  Morgan 
(1979)  found  more  positive  attitudes  among  students  using  CAI  at  the 
elementary  level  than  those  not  using  it.    Among  junior  high  school 
students  under  the  Stanford  CAI  system  of  drill  and  practice,  no  differ- 
ences were  found/  in  attitudes  compared  to  non-users  (Smith,  1973).  One 
investigation  Suggested  that  CAI  improves  the  attitude  of  students  (Hess 
&  Tenezakis,  1973)^  another  study  at  the  community  college  level  concluded 
that  CAI  d6es  not  ensure  favorable  student  attitudes  (Alderman,  1978). 

CAI  requires  students  to  interact  with  a  computer  at  a  terminal, 
thereby  providing  an  opportunity  to  exert  some  degree  of  control  over 
their/instruction.    The  Stanford,  branching  drill  and  practice  may 
encourage  persistence  (something  most  computer  users  can  empathize), 
because  it  adjusts  the  difficulty  level  so  that  students  are  always 
successful  with  some  of  the  problems.    Since  persistence  has  been  shown 
to  be  related  to  personal  control  (Lef court,  1976),  students  using  CAI 
may  develop  more  positive  attitudes  toward  personal  causation,  i.e., 
become  more  internal  about  their  successes.  ^ 

The  purpose  of  this  study  was  to  evaluate  the  differences  in  attitudes 
between  CAI  users  and  non-users.    The  curriculum  provided  drill  and 
practice  in  math,  reading,  and  language  arts.    CAI  students  were  hypothe- 
sized to  have  more  positive  attitudes  toward  math  and  reading  and  to 
indica«te  mpre  self -responsibility  or  control  of  their  successes.  ^ 

METHODS  , 

Subjects 

The  sample  of  students  for  this  study  was  selected  from  the  data 


412 


-3- 

base  of  a  l/ongi tudinal  study  of  the  effectiveness  of  CAI.     THe  students 
attended  utban  schools  designated  to  receive  aid  under  Title  I  of  che 
Elementary  and  Secondary  Education  Act  of  1965.    The  Students  were 
predominantly  from  minority  groups  (Blacks  35%,  Hispanics  44%)  and  there 
were  slightly  more  females  (55%)  than  males  (45%). 

The  sample  was  selected  because  th^se  students  had  participated  in 
CAI  fATom  their  fourth  grade  thtpugh  their  sixth  grade  and  had  relatively 
complete  data,  ^.^tude-nt^s  whose  best  language  was  not  English  were  omitted 
froija  the  study. 

In  an  earlier  study  of  achievement  putcomes  significant  negative 
correlations  between  minority  status  and  achievement  were  found  (Griswold, 
1^81).     Furthermore,  White  students  had  higher  achievement  scores  than 
^ither  Blacks  or  Hispanics.     Whites  and  minorities  were  also  split 
lineveniy,  approximately  25%  versus  75%,  respectively.     For  these  reasons 
Whites  were  omitted  from  the  study.     Finally,  some  subjects  were  lost  ' 
when  the  sample  was  selected  for  complete  data,  yielding  a  final  N  of 

126.  i  — 

]  ■  •  . 

Among  this  126  were  control  students  from  schools  and  classrooms 
from  the  same  district  or  within  the  CAI  schools.     Their  assignment^  in 
one  sensed  resulted  in  a  conservative  test  of  CAI  effectiveness  for  the 
following  reason.    The  non-j-CAI  schools  had  an  advantage  over  CAI  schools,' 
because  more  positive  classroom  observation  ratings  were  made  in  the 
non-CAI  schools  (Ragosta,  Holland,  &  Jamison,  1978).     Thus,  Students 
from  tnese  schools  came  from  better  classrooms  than  did  the  CAI  students. 

CAI  Treatment 

The  curriculum  consisted  of  drill  and  practice  exercises  or  strands 
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in  one  of  three  subject  areas:   joajfeh^  reading,  or  language.    The  range 
of  difficulty  was  four  or  five  grade  levels.    During  a  CAI  session,  a 
student  received  a  random  mixture  of  exercises  from  all  the  strands  in 
the  paixicular  content  area  appropriate  for  the  student's  grade  level,  as 
determined  at  entry.     Eighty  percent  mastery  was  the  criterion  for* 
movement  into  the  next  level  of  difficulty.    The  assignment  of  students 
to  within-CAI  treatments  was  done  at  random.    The  students  received  CAI 
for  20  minutes  per  day:     either  all  math,  half  math  and  half  reading  and 
language  arts,  or  all  reading  and  language  arts. 

Measures 

The  Iowa  Tests  of  Basic  Skills  (ITBS),  Form  5,  Level  9  were  administered 
in  the  fall,  grade  four.    Vocabulary  and  Reading  Comprehension  subtests 
were  combined  as  were  Math  Concepts  and  Problem  Solving  subtests.  The 
results  formed  a  reading  and  a  math  total  raw  score. 

A  Student  Attitude  Questionnaire,  a  variation  of  ar  instrument 
developed  by  Swinton,  Amarel,  and  Morgan  (1979),  was  administered  every 
f all  ""and  spring.     It  purportedly  measured  attitude  towards  school, 
attitude  towards  math,  reading,  and  language  arts,  self*esteem,  and  locus 
of  control.  ^ 

The  fourth  grade  questionnaire  items  were  constant  throughout  grades 
five  and  six.     This  set  formed  the  basic  instrument  for  the  present 
study.     The  scale  of  the  items  was  transformed  so  that  a  value  of  3  was 
equivaleat  to  a  positive  response,  a  2  was  an  undecided  response  and  a  1 
was  a  negative  response. 
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To  identify  th^  underlying  constructs  for  separate  evaluation,  a 
principal  components  analysis  was  performed  on  25  items  of  ttie  questionnaire  ? 
from  all  four  years  of  the  larger  study  (N  =  1659).  Those  items  that  were 
excluded  were  negative  in  orientation  (e«g«,  "Math  is  not  my  favorite 
subject/')  and  were  shown  to  have  caused  confusion  among  the  students 
(Griswold,  1981). 

Using  the  SPSS  subprogram  FACTOR,  a  five  factor  solution  explaining 
100%  of  the  variance  resulted.     In  decreasing  order  of  eigenvalue  magnitude 
the  factors  were  labeled  as  follows:    Attitude  towards  Reading,  Attitude 
towards  School,  Attitude  towards  Math,  Perception  of  the  Difficulty  of 
Schoolwork,  Perception  of  Responsibility  for  Success.     The  corresponding 

c 

eigenvalues,  rounded  to  two  decimal  places  were:    3.26,  1.52,  1.13,  .98,  .56. 

Attitude  subscales  were  formed  by  summing  items  with  factor  loadings 
greater  than  .30  under  each  of  the  factors.    The  f irst^through  third- 
subscales  corresponded  to  each  linear  composite  of  items  from  factors  one 
through  tnree.    A  fourth  subscale  was  formed  by  combining  the  composites 

♦ 

of  factors  four  and  five.  The  latter  subscale  was  redefined  as  Internal 
Responsibilt ty  for  Success  in  School. 

For  each  of  the  first  five  administrations  of  the  questionnaire  (the 
last  was  omitted  because  of  missing  data  problems),  four  subscales  of 
an  attitude  construct  were  computed  for  each  student.    Tney  consisted  of 
attitudes  toward  reading  (LREAD),  attitudes  toward  school  (LSCHO), 
attitudes  towatd  math  (LMATH),  internal  responsibility  for  success 
(CALISLPER).     Each  subscale  was  evaluated  for  internal  consistency  of  its 
items  using  the  coefficient  alpha.    The  results  indicated  moderate  to 
strong  internal  consistency  ranging  from  .47  .to  .79  with  a  median  of  .67. 
Appendix  A  includes  a  list  of  the  items  composing  each  of  the  subscales. 
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Procedures  and  Analysis 

The  tests  of  hypotheses  that  more  positive  attitudes  were  associated 
with  CAI  were  made  with  a2x2x3x5  factorial  analysis  of  variance 
(ANOVA)  with  repeated  measures  on  the  last  factor.    The  three  between 
subjects  factors  were  CAI  (Yes  or  no),  sex,  and  achievement  level.  The 
within  subjects  factor  was  attitude  (i.e.,  five  administrations  for  each 
of  the  four  subscaies).     The  achievement  factor  was  created  by  categorizing 
math  and  reading  achievement  into  three  levels:    the  upper,  middle,  and 
lower  third  of  the  raw  score  distribution  from  the  fall,  fourth  grade 
ITBS.    This  design  resulted  in  eight  ANOVAS.    One  for  math  and  one  for 
reading  achievement  by  each  of  the  four  attitude  subscaies.     The  SPSS 
subprogram  MANOVA  was  used  for  the  analyses. 

The  use  of  anaJ^ysis  of  variance  for  a  posteriori  research  is  not 
without  criticism  when  individual  difference  measures  are  categorized  as 
a  factor.    A  loss  of  statistical  power  results;  there  is  an  illusion  of 
control  of  the  variables;  there  is  a  tendancy  to  interpret  significant 
relationships  causally  (Humphreys,  1978).    The  latter  two  issues  can  be 
avoided  by  explicit  reporting  of  methods  and  results.     The  first  issue  is 
unavoidable,  but  makes  for  a  more  conservative  statistical  test,  since 
the  chance  of  rejecting  the  null  hypothesis  when  it  is  true  is  reduced. 
Given  these  caveats,  the  partitioning  of  variances  can  be  used  to  identify 
relationships;  main  effects  can  be  interpreted  as  approximations  to 
partial  correlations  (Humphreys  &  Fleishman,  1974). 

RESULTS 

The  hypothesis  that  attitude  among  students  receiving  CAI  is  more  positive 
than  among  non-CAI  students  as  supported  for  two  of  the  four  subscaies. 
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The  ANOVAS  indicated  a  main  effect  for  CAI  with  attitudes  toward  reading, 
fCl,114)  «  18:02,  £  <  .001  with  reading  achievement  levels  and  F( 1,114)  = 
17.35,  £  <  .001.    A  CAI  main  effect  also  was  found  for  internal  responsibility 
of  success,  p;(l,114)  =  8.04,  £  <  .01  with  reading  achievement  and  F(l,li4)  = 
7.89,  £  <  .01  with  math  achievement.    Tests  of  significance  within  cells 
for  CAI  using  secjuential  sums  of  squares  revealed  significant  differences 
across  repeated  measures  of  internal  responsibility,  but  not  for  reading 
attitude.    No  main  effects  were  found  for  attitude  towards  school  or 
towards  math. 

No  interactions  with  CAI  were  found,  nor  was  there  a  difference  with 

either  achievement  factor.     Thus  to  explore  further  the  nature  of  the  CAI 

I  I 

effects  t  tests  were  performed  on  unweighted  CAI  means  at  each  administration 
of  attitude  toward  reading  and  internal  responsibility.     The  results  fqr 
attitude  toward  reading  were  as  follows,. in  order  of  administration: 
£(158)  -  2.63,  _t(lb6)  -  3.87,  t_(178)  «  4.14,  ^(171)  «  3.69,  _t(156)  - 
4.24,  all  £  <  .01.     CAI  mean  attitude  toward  reading  was  significantly 
larger  than  non-CAI  means.     The  results  for  internal  responsibility  were: 
£(158)  -  1.25,  n.s.,  t(166)  «  2.00,  £  <  .05,  _t(178)  -  2.75,  £  <  .01, 
£(171)  «  2.04,  £  <  .05,  £(156)  -  4.12,  £  <  .001.    CAI  mean  internal 
responsibility  was  significantly  larger  than  non-CAI  at  all  administrations 
after  fall,  grade  four. 

Significant  main  effects  for  achievement  were  found.    When  reading 
was  the  factor  the  main  effects  were  with  attitude  towards  reading, 
£(1,114)  -  4.79,  £  <  .05  and  internal  responsibility,  F(l,114)  =  18.54, 
£  <  .001.    When  math  was^the  factor,  the  main  effects  were  with  attitude 
towards  reading,  F(l,114)  •  4.54,  £  <  .05,  attitude  towards  school, 
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F(l,114)  «  4.10,  £  <  .05,  and  ihte 
£  <  .001.  Inspection  of  the  means 
level  had  the  largest  positive  att 
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rnal  responsibility,  £(1,114)  =  19.33, 

suggested  that  the  top  third  achievement 
Itude  for  all  significant  main  effects. 


Sex  differences  across  attitude  subscales  w^re  found  only  for 
attitude  towards  school  with  either  math  or  reading  as  the  achievement 


factor.  Females  tended  to  be  more 
school. 

The  three  way  interaction  wa8< 


positive  in  their  attitudes  toward 


significant  for  attitude  towards 


school.     The  a  posteriori  evaluajticm  made  its  inte  rpretation  moot. 

: 

CONCLUSIONS  i^D  IMPLICATIONS 

s  \ 

More  positive  attitudes  toward  reading  were  found  among  CAI  students 
tiian  among  comparison,  students  in  r  on-CAI  classrooms.     This  result  may 
nave  been  due  to  a  systematic  bias,]  since  the  CAI  students'  means  were 

even  prior  to  CAI  Implementation, 
tudents  to  be  over-represented  in  the 
nay  have  contributed  to  their  lower 
D  Blacks. 


significantly  great 3r  than  non-CAI 
Cross-tabulatlons  showed  Hispanic  s 
non-CAI  gtoup.     Their  billngualism 
attitudes  toward  reading  relative  t 


A  greater  sense  of  Internal  reisponsibility  for  success  was  found 
among  CAI  students.    Prior  to  CAI  liiplementation  no  differences  were 
found  between  CAI  and  non-CAI  means^     Afterwards  students  in  CAI  had 
significantly  greater  means  for  self -responsibility^     Furthermore  the 
difference  seemed  to  widen  with  continued  CAI  exposure. 

The  lack  of  differences  with  atjtitude  towards  math  suggests  that 
20  minutes  daily  of  CAI. was  not  related  to  more  positive  attitudes.  Math 
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concepts  became  more  difficult  in  fourth,  fifth,  and  sixth  grade  and 
short  periods  of  CAI  may  not  ameliorate  dislikes.    More  lively  an  explana- 
tion is  that  the  CAI  factor  was  not  exclusively  math  drill  and  practice, 
but  included  reading  as  well.     Similar  arguments  could  be  used  for  the 
attitude  towards  reading  if  it  were  not  for  the  confounding  by  the  large 
bilingual  group  in  non-CAl. 

As  for  no  majin  effects  for  attitude  toward  school,  it  does  seem 
unlikely  that  20  minutes  daily  of  an  activity  would  make  a  difference  in 
a  general  attitude  such  as  this.    Twenty  minutes  of  confusion,  or  anxiety 
would  seem  to  neutralize  any  positive  attitudes  toward  school  generated 
by  CAI  for  its  short  duration. 

The^similarity  of  results  regardless  of  using  reading  or  math 
achievement  as  a  factor  suggests  thaV  the  ability  of  these  students  has 
not  yet  differentiated  into  levels  of  reading  and  math  ability.    That  is, 
a  good  student. is  not  selectively  better  at  math  than  reading,  but  rather 
he/she  is  good  at  all  content  areas.  , 

The  interesting  and  important  finding  with  serious  implications,  was 
that  CAI  may  prdve  helpful  to  minority  \students  by  improving  the  extent 
to  which  they  feel  in  control  of  their  successes.     By  interacting  with  a 
computer,  a  student  is  permitted  a  sense  of  control  over  the  learning 
situation  without  the  pressures  from  teachers  and  peers. 

Although  CAI  students  really  have  only  a  limited  control  of  their 
learning  session,  the  illusion  of  control  is  provided  through  logging  on, 
then  display  of  greetings  or  messages,  entering  answers  and  receiving 
reinforcing  statements,  followed  by  display  of  more  questions.  Such 
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illusion  of  control  has  been  shown  to  significantly  improve  *  performance 
in  a  variety  of  laboratory  situations  (Perlmuter  &  Monty,  1977). 

Independent  of  CAI,  attitudes  were  related  to  entry  achievement 
level  -  higher  levels  of  achievement  were  associated  with  a  greater  sense 
of  control.     This  finding  is  consistent  with  the  results  of  other  similar 
studies  (Lef court,   1976).     Sex  differences  were  anticipated  in  attitudes 
toward  school.    Females  showed  more  positive  responses. 

Generally,  CAI  curricul^  have  been  shown  to  improve  achievement. 
This  is  of  obvious  importance,  but  equally  important  is  a  neefd  for 
students  to  understand  realistically  and  objectively  their  capabilities 
(see  Smith,   1973).     From  the 'present  longitudinal  evaluation,  computer 
use  by  educationally  disadvantaged  students  may  enhance  self-responsibility 
for  academic  success.    The  reality  that  the  user  -  not  the  computer  -  is 
responsible  for  success  and  failure  may  be  a  windfall  for  students  who 
tend  to  have  less  faith  in  their  ability  to  control  the  course  of  their 
iachievement . 
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1.  Attitude  towards  Reading 

3.  Attitude  towards  Mathematics 

I  am  good  at  school 

Math  is  fun. 

I' am  very  proud  of  the  way 
'     I  read. 

I  am  good  at  math. 

I  atn  slow  at  doing  nath. 

I  am  a  good  reader. 

Math  is  the  easiest  thing 
I  have  to  do. 

I  feel  very  smart  when  c 
I'm  reading. 

I  would  rather  do  math 
than  almost  anything. 

Reading  is  easy  for  me. 

A.  Internal  Responsibility  for  Success 

2.  Attitude  towards  School 

When  I  do  well  in  language, 
it  is  usually  because 
someone  helped  me. 

I  like  school. 

I  like  spelling. 

When  I  do  well  in  language, 
it  is  usually  because  I 
tried  hard. 

*          Writing  letters  is  fun. 

When  I  do  well  at  school,  it 
is  because  the  work  is  easy. 

I  am  pretty  happy  these 
days. 

When  I  do  well  at  school,  it 
is  because  I  try  very  hard. 

I  usually  feel  pretty  good 
at  school. 

\Then  I  don't  do  my  math  very 
well,  it  is  because  the 
problems  are  too  hard. 

\ 

\ 

I  like  to  be  called  on  in 
class. 

When  I  don't  do  my  math  very 
well,  it  is  because  I  don't 
try  hard  enough. 

o 

ERIC 

\ 

■»  5 

.c 

PART  6 


COMPUTER-ASSISTED  INSTRUCTION  AND  COf'PENSATORY  EDUCATIOT^: 
THE  EFS/LAUSD  STUDY 


AssEssreiT  of  the  effectivei^ss  of 

COirUTER-ASSISTED  INSTRUCTION  IN  THE 
ETS-LOS  ANGELES  SIDDY 


GENE  V  GLASS 
UNIVERSITY  OF  COLORADO 


*This  appeared  as  Part  6  of  F1.  Ragosta^  P.  W.  Holland^  and  D.  T.  Jamison^  , 
Computer-Assisted  Instruction  and  Compensatory  Education:  The  ETS/LAUSD  Study. 
•   Final  Report  to  the  U.  S.  [^tional  Institi/te  of  Education.  April  1982. 


ERIC 


CONTENTS . 


Page 

Introduction   1 

Chapter  I       VALIDITY  OF  THE  EXPERIMENT  AND  THE  MEASUREMENTS...  2 

Experimental  Validity    2 

Measurement  Validity   9 

Over-all  Judgments  of  the  Validity  of  the 

/i                 Design  and  the  Tests   n 

Chapter  II      THE  EFFECTS  OF  ADDITIONAL  COMPUTER-ASSISTED 

I                 INSTRUCTION  .  12 

/                      One-Year  Study    12 

V/^                        The  Longitudinal  (Three-Year)  Studies   a  34 

Summary  of  Effectiveness  Analyses   37 

Chapter  III    THE  EFFECTS  OF  REPLACING  TRADITIONAL  INSTRUCTION 
BY  COMPUTER-ASSISTED  INSTRUCTION:    FINDINGS  FROM 

THE  COHORT  CONTROL  DESIGN    40 

/                             Mathematics  Achievement    41 

Reading  Achievement   44 

Language  Arts  Achievement   44 

Summary  of  Effect  of  CAI  Replacing  Traditional 

Instruction  ?  .  47 

I 

Chapter  IV     NOTES  ON  ALTERNATIVE  METHODS  OF  IMPROVING 

ACHIEVEMENT   49 

Reduction  in  Class  Size    49 

Tutoring    54 

Instructional  Television    58 

Teacher  Training   59 

Electronic  Calculators     .6f 

Summary   63 

REFERENCES    64 

i 


Part  .6 

Assessment  of  the  Effectiveness  of 
Computer-assisted  Instruction  in  the 
ETS-\Los  Angeles  Study* 

Gene  V  Glass 
l|niversiVy  of  Colorado 


In  what  follows,  an  attempt  wasV^nade  to  determine  the  impact  on 
pupils*  lea^rning  of  the  supp1ementat^on\r  replacement  of  pap^  of  their 
traditional  program  by  computer-assisted  instruction  (CAI)y/The  data 
on  which  the  estimation  of  this  impact  was  based  were  previously 
analyzed  by  Ragosta,  Holland  and  Jamison.    In  thjs  sense,  no  new  data 
were  reported  here.    However,  the  same  body  of  data  has  been  resorted 
and  reanalyzed  in  ways  that  may  make  clear  what  was  seen  less  clearly 
before  (with,  of  course,  the  concommitant  risk  that  some  things  seen 
clearly  before  are  now  lost). 

This  report  is  divided  into  four  chapters:    I.    Validity  of  the 
Experiment  and  the  Measurements,  II.    The  Effects  of  Additional  Com- 
puter-Assisted Instruction,  III.    The  Effects  of  Replee+ft§-^f aditional 
Instruction  by  Computler  Assisted  Instruction,  and  IV.    Notes  on  Al- 
ternative Methods  of  Improving  Achievement.  [ 


•   / 

*The  analyses  reported  hfere  were  produced  from  data  supplied  to 
the  author  by  the  staff  of  the  ETS/LAUSD  study.  Dr.  Marjorie  Ragosta, 
Director. 
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I.    VALIDITY  OF  THE  EXPERIMENT  AND  THE  MEASUREMENTS 

'Experimental  Valjjdity.    How  valid  was  the  ETS-LA  experimental  design, 
or  rather,  which  of  the  many  possible  experimental  comparisons  are 
most  valid? 

The  complex  and  comprehensive  design  employed  in  the  ETS-LA 
study  is  probably  familiar  to  the  reader  of  this  report.  Figure 
1  is  presented  as  ia  convenient  reminder  of  the  patterns  of  controls, 
durations  and  grades  that  make  up  the  experimental  design. 
1.    Initial  equivalence  of  groups. 

Three  types  of  control  groups  are  present  in  the  experi- 
mental design:  within-CAI  controls,  cohort  controls,  and  com- 
parison school  controls.    The  first  group  of  controls  (within- 
CAI)  were  pupils  receiving  CAI  instruction  but  not  in  the 
.subject  (Reading,  Math  or  Language)  being  evaluated.  Thus, 
a  group  of  pupils  who  studied  only  m^th  by  CAI  could  serve 
as  a  control  group  for  those  pupils  who  studied  reading  by 
CAI  when  reading  performance  was  being  evaluated.  These 
control  groups  (hereafter  referred  to  as  Within-CAI  Controls) 
have  several  highly  desirable  features:    1)  they  were  initially 
randomly  equivalent  to  the  treatment  groups,  2)  they  control 
for  novelty  effects  in .comparison  with  CAI  treatment  groups,  and 
3)  they  ^re  drawn  1[rom  the  same  classrooms  as  the  CAI  treatment 
groups  thus  being  equivalent  in  all  other  respects  (teacher  . 
I    type,  etc. )  / 
The  Cohort  Controls  were  groups  of  pupils  like  the  CAI 
treatment  groups  but  drawn  from  the  classrooms  following  the 
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Figure  1.    The  Desipn  of  cxnerirrental  Comparisons 
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treatment  groups  by  one  year.    In  spite  of  the  many  ways  in  which  it 
can  be  argued  that  they  should  be  jsimHar  to  the  CAI  treatment  groups, 
1t  Is  difflcuTIt  to  judge  the  diffejrence  a  year  makes;  hence,  their 

IhitiaV  equivalence  to  the  CAI  tre'atment  groups  is  problematic. 

• ! 

•    The-third  control  condition  comprises  pupils  in  different  schools 
altogether.    The  objections  to  its  use  as  a  control,  are  many  and 
obvious. 

2.    Accessibility  and  Attrition. 

Over  a  period  of  either  one  or  three  years,  pupils  left  the- 
program.    The  reasons  for 'leaving  are  unclear;  it  is  unlikely  that 
the  left  because  of  the  kind  of  CAI  treatment  they- received.    It  is 
highly  implausible  that  they  would  leave  a  schooVL^ue  to  the  type  of 
CAI  groups  (Math-Math  vs.  Reading-Language  vs.  Readin|-Reading)  they 
were  assigned  to.    In  advance  of  inspecting  the  data,  I  would  have  bet 
that  attrition  would  have  been  strictly  randomly  distributed  among  the 
CAI  curriculum  groups,  leaving  the  validity. of  the  Within-CAI  Control 
groups  uncompromised.    However,  the  pre^test  means  of  the  CAI 
curriculum  groups  differ  significantly  appearing  to  indicate  that 
attrition  was  not  random.    For  example,  consider  the  CTBS  Math  Pre- 
test scores  for  Grade  6  students.  Longitudinal  Data,  Total  Math: 

CAI  Group 

Math-Math  MRL  Reading-Lang, 

n:  32  34  28 

X.:  57.44  58.88  68,21 

s^:  24.00  22.14  18.52 


The  ANOVA  F-ratio  is  2.33  with  df  =  2.91;  its  percentile  in  the 
central  F-distribution  is  approximately  the  90th;  thus,  the 
probabiTltytfiat  ra  created  discrepancies  as  Targe  as 

these  is  only  about  .10.    The  F-ratio  for  the  Reading  pretest  scores 
is  2.95,  which  is  nearly  significant  at  the  ,05  level. 

In  fact,  at  least  two  factors  were  at  work.    During  the  first 
year  of  the  study,  nonreaders  and  Spanish-speaking  students  who  could 
not  understand  the  reading  curriculum  were  not  forced  to  continue 
that  assignment  but  were  removed  from  the  assigned  CAI  group  and  re- 
assigned to  Mathematics.    Their  data  did  not  appear  in  the  study's 
results.    In  addition,  attrition  over  three  years  operated  differently 
on  the  MM,  MRL  and  RL  groups:    MM  and  MRL  means  on  the  inital  pretest 
decreased  over  time  and  RL  means  increased  over  time.    Of  the  two 
factors  the  greater  contributor  to  initial  differences  was  the  first 
factor.    One  has  little  alternative  but  to  take  the  fact  of  non-random 
accessibility  and  attrition  seriously  and  worry  about  it.    The  details 
of  this  concern  appear  in  the  following  section. 

3.    Ex  Post  Facto  Correction. 

Inaccessibility  and  non-random  attrition  of  pupils  ^across  the  CAI 
curriculum  groups  may  have  turned  an  unusually  well  designed  experiment 
into  a  slightly  problematic  one.    Where  random  assignment' was  never 
attained,  or  broke  down,  or  was  compromised  by  differentially  mortality 
among  treatment  groups,  the  analysis  of  data  falls  into  the  gray  area 
of  ex  post  facto  analysis  of  covariance  adjustment.    From  all  that  is 
currently  understood  about  the  problem  of  attempting  to  correct  the 
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faults  of  non-randomness  by  covariance  adjustment  (Cronbach  et  aj_. , 
1977),  one  takes  on  these  problems  with  deep  regrets.    There  is  no 
genuinely  satisfactory  solution,  and  analysis  of  covariance  may  unde 
correct,  overcorrect  or  correct  perfectly  depending  on  ^  variety  of 
conditions  (measurement  reliability,  discrepancies  among  groups  on 
unobserved  variables)  about  which  one  has  typically  no  information. 
With  the  ETS-LA  study,  we  find  ourselves  in  such  circumstances. 
I  am  inclined  to  take  the  analysis  of  covariance  findings  at 
face  value,  but  they  are  worrisome  precisely  because  they  must 
be  regarded  as  an  ex  post  facto  patching  up  of  a  randomized 
experiment  that  could  not  be  maintained  as  fully  randomized. 
In  the  end,  perhaps,  it  is  the  findings  and  estimates  of  effects 
themselves  that  are  the  best  evidence  of  their  validity  because 
they  form  patterns  consistent  with  the  interpretation  that  the 
method  (CAI)  was  effective  and  that  the  Within-CAI  control 
design  assessed  its   effects  reasonably  validly. 

The  validity  of  the  Treatment  vs.  Cohort  Control  design 
is  even  more  problematic  than  that  of  the  Within-CAI  Control 
design,  for  the  reason  that^ assignment  between  CAI  and  Cohort 
Control  groups  was  non-random  in  unknown  ways.* 
/  4.    Control  of  Intructional  Time. 

In  most  respects,  the  Within-CAI  comparison  is  the  most 
internally  valid  experimental  comparison  available.    But  some 
have  objected  that  such  a  comparison  does  not  properly  represent 
the  choice  between  CAI  and  traditional  instructional  methods 

*Desi gnat ion  of  CAI  vs.  control  years  for  cohort  controls  was  somewhat 
arbitrary  and  might  be  regarded  "as  if  "  random;  but  even  so,  it  was 
as  if  random  at  the  school  level  at  best  and  not  at  the  level  of  pupils. 


faced  by  the  practitioner. 

The  consideration  of  this  question  opens  up  a  deeper  issue, 
which  is  best  approached,  perhaps,  through  a  description  of  the 
schedule  of  activities  in  a  typical  CAI  classroom.    Students  were 
randomly  assigned  to  one      the  CAI  curriculum  groups  (viz.,  M,  MM, 
RL,  MRL).    For  a  short  p4riod  several  times  a  week,  either  the 
entire  class  or  a  port/on  of  it  was  sent  to  the  CAI  lab.    When  only 
a  portion  of  the  class  was  thus  pulled  out,  those  pupils  who 
remained  might  study  an  academic  subject  or  might  engage  in  some 
less  academic  activity;  exactly  what  they  did  varied  from  classroom 
to  classroom  and  probably  varied  from  week  to  week  within  the  same 
classroom.    In  addition,  all  pupils  were  taught  in  basic  subjects 
as  a  normal  part  of  classroom  work.    An  inspection  of  teachers' 
lesson  plans  would  show  several  hours  per  week  of  traditional 
classroom  instruction  devoted  to  reading,  math  and  language. 
Hence,  the  experiment  must  be  regarded  as  a  study  of  CAI  supplementing 
traditional  classroom  instruction.  (This  point  is  important  since  it 
may  not  be  reasonable  to  conclude  that  a  successful  supplement  would 
be  a  successful  steac(y  diet;  boredom  effects  would  have  to  be  con- 
sidered, and  particularly  in  the  case  of  CAI,  some  of  its  benefits 
may  depend  on  the  opportunity  to  ask  questions  of  human  teachers  that 
the  machine  can  not  answer.) 

What  kinds  of  instruction  did  pupils  in  various  CAI  curriculum 
groups  experience?   One  can  speculate  with  reasonable  confidence. 
Suppose  a  teacher  taught  normally  and  sent  the  entire  class  to  the 
CAI  lab  for  15-20  minutes  several  times  a  week.    Then  each  pupil 
(regardless  of  CAI  curriculum  group)  would  receive,  perhaps,  100-150 


hours  of  reading  instruction  in  a  school  year  and  about  75-100.  hours 
of  math  instruction.    In  addition,  pupils  in  the  MM  strand  would 
receive  about  15-20  min.  daily  of  CAI  instructibh  oh  top  of  classroom 
math  Instruction  and  more  than  pupils  in  the  RL  strand.    Thus,  with 
respect  to  math  ir^struction ,  we  have  the  following  situation: 


Hrs./Yr.  of  Math 
Instruction  in 
/  Traditional 


Hrs/Yr.  of  Math 


Total 
Math  Instruction 


Pupils  in 
CAI  MM  Strand 


Pupils  in 
CAI  RL  Strand 


90  hrs. 

50  hrs. 

90  hrs. 

90  hrs. 

■  — — 

0  hrs. 

90  hrs. 

Under  these  circumstances,  the  CAI  MM  group  receives  156%  more 
instruction  in  math  than  the  CAI  RL  group,  which  serves  as  its  control 
group  in  the  Within-CAI  control  group  design.    Clearly,  under  these 
circumstances  the  comparison  of  CAI  MM  with  CAI  RL  on  mathematics 
achievement  must  be  regarded  as  an  assessment  of  the  effects  of 
additional  math  instruction  provided  by  computer  over  and  above 
instruction  in  math  in  the  regular  classroom.    If  a  decision  maker 
were  considering  replacing  school  time  spent  in  art  or  physical 
education  with  CAI  math  instruction  over  and  above  the  school 
normal  program,  this  experimental  comparison  would  speak  to  his 
decision. 

However,  CAI  MM  vs.  CAI  RL  would  not  speak  to  the  question 
whether  pupils'  time  spent  in  the  CAI  lab  studying  math  is  better 
spent  there  than  in  the  classroom  receiving  traditional  math 
instruction  from  their  regular  teacher,  "fo^ahswer  this  question, 
one  must  focus  on  tha  CAI  vs.  Cohort  Control  comparisons. 
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Which  quest! on, ought  one  to  answer:  'Traditional -pi us-CAI  vs. 
Traditional  or  Traditional-and-CAI  vs.  Traditional  (implying  that 
net  instructional  time  is  equated  in  the  latter  comparison)?  There 
is  no  single  answer.    But  clearly,  it  can  not  be  argued  that  one  of 
these  comparisons  is  proper  and  the  other  improper.    It  is  easy  to 
imagine  a  school  district  contemplating  the  implementation  of  CAI 
either  as  a  means  of  increasing  total  time.spent  in  basic  skills 
TfTTtruction  or  as  a  replacement  for  some  part  of  that_time  currently 

spent  in  traditional  classroom  instruction  in  basic  skills.    In  the 

\  /  ^ 

former  ckse,  the  Within-CAI  Cdntrol  cotrparison  will  be  most  relevant; 
in  the  latter  case,  the  CAI  vs.  Cohort  Control  comparison  will  be 
most  relevant. 

B.    Measurement  vkl i di ty.    Which  outcome  measure  is  better  (i.e.,  mdre 
"valid,"  falrerXto  each  program  (CAI  or  Traditional),  more  relevant 
to  the  value  thatV teachers  and  parents  feel  they  are  purchasing 
fromjth^  schools)?^ 
L         The  CTBS  is  a  wfell-known  survey-type  test  of  achievement  in 
basic  skills  (readings  math,  language  arts).    The  CST  (Curriculum 
Specific  Test)  was  constructed  by  the  ETS  project  director  from 
among  the  examples  and  exercises  in  the  computer  so/t-ware  for 
the  CAI  course.    In  the  sense  of  practice  on  item  forms  (i.e..  the 
obvious  features  of  items  such  as  horizontal  vs.  vertical  addition), 
the  CST  favors  the  CAI  subjects  and  might  even  be  said  to  be  'biased" 
in  favor  of  the  CAI  group.    For  example,  early  item  forms  on  the  CST 
represented  fractions  in  the  form  2/3,  a  format  dictated  by  the 
computer  harcware  but  unfamiliar  to  control  pupils  who  had  been 
taught  to  deal  with  fractions  in  the- form  2  .  This  problem  was 
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detected  and  corrected^n  a  subsequent  revision;  but  the  possibility 
remains  that  less  obvious  superficial  features  remain  in  the  CSTs 
and  continued  to  favor  the  CAI  group  in  a  way  that  was  unfair. 

But  charges  of  "bias"  are  not  to  be  made  lightly.  Sometimes 
a  "mere"  matter  of  fom  in  a  curriculum  actually  represents  improved 
pedagogy,  and  to  remove  it  is  to  unfairly  deprive  the  curriculum  of 
one  of  its  comparative  advantages.    So  a  judicious  course  must  be 
steered  between  calling  a  test  "biased"  in  favor  of  one  curriculum 
versus  reducing  test  coverage  to  the  common  core  of  items  that  may 

.7  i. 

fail  to  credit  a  curriculum  with  its  unique  accomplishments. 

I  have  not  been  able  to  perform  a  careful  content  analysis  of 
the  CSTs  and  the  curricula  in  the  present  study;    But  based  on  a 
knowledge  of  the  history  of  their  production,  I  am  inclined  to  feel 
that  the  findings  on  the  CSTs  overestimate  the  genuine  value  of  the 
CAI  program/  On  the  other  hand,  the  CTBS  findings  run  a  slight 
risk  of  underestimating  the  benefits  of  the  CAI  program.  Survey- 
type  standardized  tests  like  the  CTBS  often  fail  to  reflect  accu- 
rately  the  content  of  the  curriculum  of  a  single  school  district; 
hence,  their  use  in  evaluating  a  new  teaching  method  in  a  particular 
district  may  tend  to  underestimate  fts  value.    This  problem  could 
be  serious  in  subjects  "like  social  studies  where  curricula  differ  = 
greatly  among  school  districts;  it  should  be  less  severe  in  reading 
and  math,  particularly  at  earlier  grades.    Language  arts  curricula 
might  be  expected  to  differ  more  among  districts  than  reading  and 
math  curricula.    Thus,  the  CTBS  can  be  expected  to  be  more  adequate 
in  evaluating  the  effects  of  CAI  on  reading  and  math  achievement  than 
on  language  arts  achievement. 


The^relevance  of  validity  of  the  various  measures  of  outcomes  of 
instruction  might  be  described  in  the  following  terms. 


Type  of  Test 


-      CTBS      .  ' 

CST 

Math 

well -matched  with 
Math  beyond  the 
. CAI  curriculum. 

well -matched  with 
particular  curriculum/,: 
less  representative  of 
Math  more  generally. 

)ject 

Reading 

well -matched  with 
Reading  beyond  the 
CAI  curriculum. 

well -matched  with 
particular  curriculum; 
less  representative  of 
Reading  more  generally. 

3 

CO 

Language 

only  moderately 
well-matched  to 
both  particular 
curriculum  ^nd 
Language  Arts, 
more  generally. 

well -matched  with 
particular  curriculum; 
less  representative  of 
Language  Arts  more 
generally. 

C.    Over-all  Judgments  of  the  Validity  of  the  Design  and  the  Tests. 


For  answering  the  question  whether  CAI  is  beneficial  when  it  adds 
to   the  regular  program  of  instruction  of  the  school,  the  Within-CAI 
Control  design  is  most  relevant.    For  answering  the  question  whether 
CAI  is  beneficial  when  used  in  place  of  part  of  the  regular  instruc- 
tion time,  the  cohort-control  design  is  most  relevant.    Although  the 
former  design  was  randomized  but  appeared  to  suffer  from  differential 
attrition  (with  poorer  students  dropping  out  of  the  RL  and  MRL  curri- 
culum groups)  and  although  the  latter  design  was  not  randomized,  their 
findings  after  analysis  of  covariance  adjustment  on  pretests  are  the 
best  evidence  available  to  answer  the  questions.    Data  from  the 
Comparison  School  design  will  not  be  used. 
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The  Curriculum  Specific  Tests  were  constructed  in  such  a  way  that 
CAI  students  probably  enjoyed  an  advantage  on  them  for  reasons  unrelated 
to  genuinely  superior  achievement.    The  Comprehensive  Test  of  Basic 

Skills  shares  the  well-known  faults  of  standardized  tests  for  curri- 

c 

0 

culum  evaluation;  nonetheless,  it  is  preferable  to  the  CSTs  in  this 
instance  and  will  be  used  exclusively  in  subsequent  analyses  of  the 
ETS-LA  experiment. 

II .    The  Effects  of  Additional  Computer-Assisted  Instruction 
A.     One-year  Study 

Analyses  in  this  section  will  be  confined  to  data  from  "one-year" 
(i.e. /not  longitudinal)  experimental  comparisons.    All  comparisons  are 
of  the  Wlthin-CAI  control  group  type.    The  pattern  of  experimental 
comparisons  is  complex;  they  come  from  different  years  and  are  made  at 
different  grades.    They  are  shown  in  Table  1. 

1.    Mathematics  Achievement.    In  Table  1  appear  the  experimental 
comparisons  by  grade  and  year  in  which  double  math  .periods  (MM)  were 
compared  with  single  (M)  or  in  which  math  Instruction  (either  MM  or  M) 
was  computed  with  instruction  in  reading  (R),  language  (L)  or  both. 

i 
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;   Table  1 
Experimental  Comparisons  of  CAI 
Instrucffdn  in  Math  (M).  Reading  .{|)  or  Language  (L) 


MM  V.  ML  M  V.  L 

MM  V.  RL       M  y.  R  or  L 

MM  V.  RL  MM  V.  RL 

MRL  V.  RL 

MM  V.  RL 
MRL  V.  RL 


.      *Instruction  in  Year  1  lasted  for  only  1/4  school  year. 

Letting  v  represent  the  effect  of  one  unit  (10  mins/day)  of  CAI 
math  instruction  a^r^n  represent  the  effects  of  two  units  (20  mins/day)- 
it  can  not  be  asulmed  that  2vi  =  n  —  then  an  experimental  comparison 
such  as  M  V.  L  estimates  v  and  a  comparison  such  as  MM  v.  M  estimates  _^ 
n  -  y.    In  Table  2  appear  the  parameters  estimated  by  each  experimental 
comparison. 
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Table  2 

Parameters  Estimated  by  the  Experimental 
tomparisons  in  Table  V 


Year 

1*  2 

3 

'  '  ^  / 

Grade 

/ 

2 

Tl  -  P  (?),* 

3 

n  -  p 

.  4 

n 

5 

n 

p 

6 

n 

\ 

\ 

\ 


.  V 

♦Instruction  in  Year  1  lasted  only  1/4  school  year.  \ 

-  The  empirical  findings  were  treated  in  -the  foTlowing  manner.-  In 

V 

each  experimental  comparison,  the  pretest-covariance  adjusted  postte^t 

\ 

average  for  the  curriculum  group  had  the  adjusted  posttest  average  fof 
the  within-CAI  control  group  subtracted  from  it  and  the  difference  was  - 
standardized  by  the  within  classroom  among  pupil  standard  deviation 
(after  removing  variance  due  to  sex)\^ethnic  group  and  classroom). 

Adjusted  55^,ea4e,t  '  ^'^j^^^^'^  '^Control 


The  considerations  that  bear  on  this  choice  of  a  measure  for  expressing 
the  findings  of  the  experiment  are  severaV  and  complex;  they  are  discussed 


in  Glass,  McGaw  |&  Smith  (1981).    The  measure  A  describes  the  average 
diffe'rence  in  e)^erimental  and  control  groups  measured  in  units  of  the 
standard  deviation  of  pupils'  scores.    The  meaning  of  its  algebraic 
sign  (+  or  -)  is  obvious;  assuming  normal  distributions  of  scores 
(probably  a  safe  assumption  in  the  present  instance),  its  magnitude  can 
be  interpreted  in  terms  of  the  well-known  properties  of  the  normal 
distribution,  e.g.,  if  A  =  +1 ,  then  the  average  pxipiTs  score  in  the 
experimental  group  exceeds  84%  of  the  pupils'  scores  in  the  control 
group,  and  so  forth.    A  specfal  meaning  that  can  be  attached  to  a 
standardized  mean  difference  measure  of  effect  size  in  educational 

r 

contexts  derives  from  widely  known  norms  for  the  rate  of  achievement 
growth  per  month  of  typical  schooling.    At  most  elementary  school 
grades  for  most  measures  of  educational  achievement,  the  difference 
between  the  average  pupil  at  the  beginning  of  a  grade  and  at  the  end 
of  the  same  grade  is  about  1.0  standard  deviation  units.    Hence,  one 
month's  typical  instruction  (or  "time  spent  in  school")  accounts  for 
a  growth  of  about  0.10  standard  deviation  units.*  Hence,  an  Instructional 
method  that  produces  a  +.50  effect  size  is  responsible  for  benefits  equal 
to  those  bought  by  five  months  of  typical  classroom  instruction. 

The  estimates  of  parameters  appear  in  Table  3  for  the  CTBS  mathe- 
matics test,  which  included  subtest  scores  for  Computation,  Concepts 
and  Applications. 
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Table  3 

Estimates  of  Parameters  for  CTBS  the  Subtests  Math 
Computation/Math  Concepts/Math  Applications. 


.  Year 


Grade 
2 
3 
4 
5 


n  -  m:  .35/. 33/-*  / 


T\  -  u:  .38/-. 03/. 04 


n:  .27/. 08/. 10 


n:  .37/-. 02/-. 09 
m:  .19/-. 27/. 04 


m:  .27/. 06/. 10 
m:  .43/. 07/. 04 

n:  .52/-. 10/-. 16 


,33/. 05/. 26 
,16/-. 09/. 21 


*^CTBS  Concepts  &  Applications  subtests  are  combined  at  Grade  2. 
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Table  3  can  be  read  as  follows:    in  Grade  3  during  Year  2  (in 
which  MM  was  compared  to  M) ,  the  covariance  adjusted  standardized  mean 
difference  between  MM(20  min./day)  and  M  (10  min./day)  of  CAI  instruc- 
tion is  .38  units  on  the  Computations  subtest,  -.03  units  on  the 
Concepts  subtest  and  .04  units  on  the  Applications  subtest.    Thus,  in 
math  computation,  the  average  third-grade  pupil  who  received  20  minutes 
daily  of  CAI  instruction  scored  higher,  by  .38  standard  deviation  units 
than  the  average  pupil  who  received  10  minutes  of  CAI  math  instruction 
daily.    In  math  concepts,  the  average  M  pupil  scored  higher  than  the 
average  MM  pupil,  but  trivially  so  (by  .03  standard  deviation  units). 

a)  Math  Computation.    The  various  estimates  of  y  and  n  in  Table  3 
can  be  combined  to  obtain  over-all  estimates. 

The  average  of  all  the  four  different  estimates  of  y  in  Table  3 
is  +.29.    The  four  estimates  of  n  average  is  .37.    Combining  these 
yields  an  estimate  of  n  -  y  of 

n  -  y  =  .37  -  .29  =  +  .08. 

But  in  the  two  instances  in  which  MM  and  M  were  compared  directly 
within  the  same  classrooms  (Grade  2,  Year  1  and  Grade  3,  Year  2),  the 
estimate  of  n  -  y  =  (.35  +  .38)/2  =  .36. 

These  estimates  (.08  vs.  .36)  derived  from  two  separate  sources 
are  somewhat  disparate.    Perhaps  the  discrepancy  results  in  part  from 
a  confounding  of  treatment  effects  with  grade  level,  since  the  .36 
estimate  of  n  -  y  comes  from  data  at  Grades  2  and  3  and  the'  .08  estimate 
from  data  at  Grades  4,  5  and  6.    The  discrepancy  can  not  be  attributed 
confidently  to  such  an  interaction  since  the  entire  experimental  design, 
of  necessity,  fell  far  short  of  ^  completely  crossed  factorial  design.. 


\ 

There  appeaj\^  to  be  no  "year"  effects  in  the  data  for  math  computa- 

"  / 

tion,  i.e.,  experiments  in  years  3  and  4  don't  Appear  to  have  given 

/  ■     ;  ■ 

systematically  different  estimates  from  those  done  in  years  1  and  2. 

/  /        •  ■      '    '  ■ 

But,  again,  in  so  feir  as  can  be  discerned  in  the  incomplete  design  of 

Table|2,  there  apf^ears  to  be  an  interaction  of  MM  v.  M  comparisons  and  " 

grad^  level.    Hence,'  in  what  follows  immediately,  n  and  y  will  be 

estimated  separately  for  grades and  3  and  for  grades  4,  5  &  6. 

The  problem  of  what  to  do  with  the  Yeaf  1  data  for  Grade  2  is  difficult 

/  / 

to  resolve.    If  instructional  time  is  lin^ar^y  related  to  amoiint  learned, 
then  perhaps  the  figures  for  Year  1 /should  be  quadrupled  si^e  CAI 
instruction  was  given  only  lot  the/months  of  February  through  April, 
or  at  least,  the  CTBS  was  administered  in  April.    But  multiplying  .35 
by  4  gives  an  absuredly  large  figure  of  1.40  for  the  advantage  of  MM 
.'over  M,  a  figure  that  is  much  larger  than  any  other  direct  estimate 
of  n  -  y  for  a  whole  school  year.    It  is  difficult  to  know  how  quickly 
pupils  moved  through  materials  in  the  quarter  year  of  CAI  in  Yea^r  1; 
failing  that  knowledge  and  faced  with  the  embarrassingly  large  1.40, 
it  seems  wiser  to  take  the  .35  estimate  at  face  value  and  not  tamper 
with  it. 

From  all  data  in  grades  2  and  3.  the  one  available  estimate  of  y 
is  +.27.    The  two  estimates  of  n  -  y  agree  quite  closely:    .35  and  .38; 
it  seems  sensible  to  replace  them  with  their  average,  .36.    The  two 
estimates  have  the  following  implications) 

n  -  y  =  .36  ' 
y  =  .27, 

imply  that 

n  =  .36  +  y  =  .36  +  .^7  =  .63. 
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Hence,  at  Grades  2  and  3,  the  effect  of  10  min/day  of  CAI  on  math 
computation  is  .27  standard  deviation  units  on  the  average;  the  effect 
of  20  min/day  is  .63  standard  deviation  units. 

At  grades  4,  5  and  6,  the  results  look  slightly  different.  The 
three  available  estimates  of  y  (the  effect  of  10  min/day)  average  .26. 
The  four  estimates  of  n  (20  min/day)  agree  somewhat  poorly 
(.27,  .33,  .37,  .52)  and  their  average  is  .37.    Combining  these  two 
estimates  yields  an  estimate  of  the  effect  of  the  second  daily  10  min 
Instructional  period  of 

n  -  i  =".37  -  .26  =  .11. 

But  an  argument  could  be  made  that  the  Grade  5,  Year  2  and  Grade 
6,  Year; 3  experiments  are  to  be  preferred  in  estimating  n  -  y  since  MM 
and  M  were  applied  to  the  same  classrooms  within  each  experiment. 
Basing  the  estimate  only  on  these  two  experiments  yields 
n  -  y  =  (.37  +  .33)/2  -  (.19  +  .16)/2  =  .18. 
This  figure  agrees  nicely  with  the  over-all  estimate. 

Hence,  at  Grades  4,  5  and  6,  the  effect  of  10  min/day  , CAI  on  math 
computation  is  .26  standard  deviation  units  on  the  average;  the  effect 
of  20  min/day  is  .37  standard  deviation  units. 

Although  the  effects  of  10  min/day  of  CAI  instruction  is  greater 
at  Grades  2  and  3  than  at  Grades  4,  5  and  6,  at  either  level  the 
effect  of  20  min/day  is  from  one  and  one-half  to  tw*ice  as  large  as  the 
effect  of  10  min/day. 

b)  Math  Concepts. 

There  is  no  evidence  of  any  treatment  by  grade  level  interaction 
on  achievement  in  math  concepts,  so  the  analyses  that  follow  immediately 

4 1 7 
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will  ignore  grade  level.    Grade  2  in  Year  1  is  eliminated  since  math 
concepts  and  applications  were  not  tested  separately. 


Type  of  Estimate  , 

Value 

Grade 

Year 

.06 

3,  •  • 

4 

-•27 

.  5 

2 

u 

-.09 

6- 

3 

n 

.08 

4 

3 

n 

-.02 

5 

2 

n 

•.10 

/ 

5 

4 

n 

/ 

.,05 

6 

3 

^  n  -  y 

7.03 

3 

2 

These  effects  are  consistently  small,  and  worse,  mostly  negative 
The  estimates  of  n  and  y  car/ be  formed  by  simple  averaging  of  the 
individual  estimates:  / 

u       =  -.10 
n     .=    .00  . 
The  difference  n  -  u  estimated  in  this  manner  is 

n  -  u  =  .00  -  (-.10)  =  +.10. 
The  direct  experir^ental  estimate  of  n  -  u  is  -.03.    The  average 
of  these  two  estimates  is  +.03. 
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Th(jse.  si  ight  ambiguities  can  be  cleared  up  by  forming  a  set  of 
simultaneous  linear  equations  that  can  be  solved  by  methods  of  least 
square  estimation,  as  follows: 


1 

1  : 

1 

0 

0 

0 

0 


.oel 

-.27 

-.09  I 

.08  i 

-.02'  j 

-.10  1 

.05  \ 
-.03  J 


Denoting  the  left-most  matrix  by  _X  and  the  right-most  by  X.  the 
estimates  are  obtained  from 


x'''y, 


The  solution  is  as  follows; 


-.07 


.02 


These  results  can  be  taken  in  one  of  at  least  two  ways:    (1)  the 
CAI  math  instruction  for  10  min/day  confuses  (p    =  -.07)  pupils  at  the 
conceptual  level  but  the  confusion  is  removed  with  another  10  min/day 
so  that  they  are  left  where  they  started  (n    =  -.02);  or  (2)  the  CAI 
math  instruction  has  no  added  benefits  at  all  at  the  conceptual  level 
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and  the  small  numbers  (-.07,  -.OZ)  merely  reflect  random  or  noisy 
perturbations  in  the  evaluation  system.    I  favor  the  second  interpreta- 
tion. 

c)  Math  Applications    .  . 

The  e'^.t "inflated  effects  and  the  parameters  estimated  for  math 
applications  are  as  foTTpws: 


Type  of  Estimate 


Value 


Grade 


Year 


3 

4 

y 

.04 

5 

2 

y 

.21 

6 

3 

n 

.10 

4 

3 

n 

/  -.09 

5 

2 

n 

-.16 

5 

4 

n 

.26 

6 

3 

n  -  y 

■  .04 

3 

2 

Again,  this  set  of  numbers  and  parameter  estimates  can  be  treated 
as  a  system  of  linear  equations  and  solved  by  the  methods  of  the  previous 
section:  ,  • 

.07 


(X^X)"^  X^Y 


,04 


Thus,  the  effect  of  CAI  instruction  in  math  on  math  applications 
is  essentially  zero  whether  given  for  10  min/day  or  20  min/day. 
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d)    Sumnary  of  Findings  for  Mathematics. 


Covariance  Adjusted  Effect  Sizes 
Classified  by  Type  of  Math  Achievement,  - 
Grade  Level  &  Amount  of  Instruction 


Area  of  Math  Achievement 


Computation  Concepts  Applications 

Amount  of 

CAI  Instruction        Grades    Grades  Grades    Grades  Grades  Grades 

per  day  '                 2&3      4,5&  6  2&3      4,5&  6  2&3      4,5&  6 


^  10  mins.  .27        .26  -.07        -.07  .Q7  .07 

20  mins.  .63        .37  -.02        -.02  .04  .04 


2.    Reading  Achievement. 

The  design  of  one-year  experiments  assessing  CAI  reading 
instruction  with  an  off-strand  instructional  program  is  presented 
>in  Table  4. 

Table  4 


Experimental  Comparison  for 
CAI  Reading  Instruction 


Year 


Grade  1' 


4  MRL  V.  MM  RL  v.  MM  R  v.  M 

RL  V.  MM 

5  .        MRL  V.  MM  RL  v.  MM 

RL  V.  MM  • 

6  '    RR  V.  m  MRL  V.  MM 

RL  V.  MM  RL  V.  MM 


*  In  Year  1,  instruction  lasted  only  one  quarter  of  the  school  year 
(Feb.  -  April). 
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^If  'p   Stands  for  the  effect  of  10  min/day  of  CAI  reading  instruction 

and  6  is  the  effect  of  20  min/day,  then  the  experimental  comparisons 

in  Table  4  provide  estimates  of  these  parameters  as  indicated  in 

Table  5.    Note:    In  the  MRL  strand,  reading  instruction  is  for  a  * 

total  of  25  minutes  per  week  or  5  minutes  per  day.    Denote  the  effect 

of  5  fnin/day  of  CAI  reading  instruction  by  ip. 
^  Table  5     ,  / 

Parameters  Estimated  by  Experimental 
Comparisons  in  Table  4  ,, 


Year 


Grade 

4  }l>  '  p 
P 

5  ■  * 

P 

6  ®  * 
P  P 


The  estimates  of  the  effects  of  these  parameters,  again  in  the  form 
""of  covariance  adjusted  standardized  mean  differences,  appear  in  Table' 
6  for  reading  vocabulary  and  comprehension. 
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Table  6 

Effect  Sizes  from  Experimental  Comparisons  (Table  4)  and 
the  Parameters  Estimated  (Table  5)  for  Reading  Vocabulary/ 
Reading  Comprehension. 


Year 

Grade  1  2 


i:  .04/-. 04  p:  .25/. 26  p:  .26/. 18 

I:  .28/. 23 

i:  .03/. .25  p:  .23/. 31 

p:  -.03/-. 09 
e:  .26/. 45  i:  .51/-. 06 


P 


.28/. 51  .  p:  .45/-. 03 


a)    Reading  Vocabulary 

There  are  seven  estimates  of  the  effect  of  10  min/day  CAI  instruc- 
tion on  reading  vocabulary  in  the  data  in  Table  6.    When  averaged, 
these  estimates  show  the  fol lowing  trends  across  "years  and  grades: 

Average  Estimates  of  p  By 
Year 

1  2  3  -4 


.34     ■    -.03  .35  .24 
Grade 

4  5  6  - 

.26       .10  .37 
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There  are  differences  to  be  noted  here,  viz.,  all  the  way  from 
-.03  to  .37,  but  the  differences  do  not  look  like  reasonable  trends. 
The  "best  year"  for  CAI  reading  vocabulary  was  the  third,  and  the 
"best  grade"  is  the  sixth,  but  the  fifth  is  worse  than  the  fourth. 
The  variability  is  more  confusing  than  orderly  and  conclusions  about 
interactions  of  CAI  effectiveness  with  grade  level  or  year  of  the 
study  would  be  unwise.    The  variability  of  effects  of  CAI  on  reading 
vocabulary  in  Table  6  is  distressingly  large  (from  -.03  to  +.51)  and 
can  not  be  accounted  for  by  anything  simple  arid  obvious. 

The  average  of  the  three  estimates  of     is  +.^9,  but  to  obtain  it 
one  had  to  average  .04,  .03  and  .51  (a  case  of  the  average  not  being 
very  "average").  / 

Averaging  the  seven^stimates  of  p  in  Table  6,  we  obtain 

p  =  .25. 

The  single  available  estimate  of  the  effect  of  20  min/day  CAI 
instruction  on  reading  vocabulary  (from  Grade  6  -  Year  1)  is 

0  =  .26. 

Is  10  min/day  of  CM  instruction  really  as  effective  as  20  min/day, 
-as  reflected  by  p  =  .24  and  6  =  .26?    I  doubt  it.    The  estimate  of  e 
is  even  more  dubious  because  it  came  from  a  Year  1  experiment,  the 
• '^year"  that  lasted  only  from  February  to  April.    (But  to  add  to  the 
'confusion,  notice  that  p  for  Grade  6  -  Year  1  is  larger  than  the  average 
of  all  other  estimates). 

I  am  inclined  to  treat  the  single  20  minute  (RR)  experimental 
comparison  and  the  three  5  minute  (MRL)  comparisons  just  like  the  other* 
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10  minute  (R)  comparisons  and  average  all  11  estimates  of  the  effect  of 
CAI  on  Reading  vocabulary. 

Conclusion:  p  =  .23  for  reading  vocabulary,  with  the  qualification 
that  the  estimates  of  experimental  effect  are  quite  variable  around  this 
average  figure,  ranging  from  -,03  to  -f.Sl. 

b)  Reading  Comprehension 

The  reasoning  that  led  to  the  estimate  of  p  for  reading  vocabulary 
applies  in  many  respects  to  the  estimation  of  the  effect  of  CAI  instruc- 
tion on  reading  comprehension.  '  ^ 

The  estimates  of  p  for  reading  comprehension  in  Table  6  range 
from  -.25  to  +.51,  and  thus  are  even  more  variables  than  the  estimates 
of  the  effect  on  reading  Vocabulary.    The  inspection  of  the  data  for 
trends  across  years  and  grades  produces  the  following  results: 

Average  Estimates  of  p  By 
Year 

1  2  3  4 

.29     -.17       .08  .25 
Grade 
4  5  6 

.16     -.01  .23 
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\      No  sensible  trends  for  either  Year  or  Grade  Level  are  apparent. 
This  three  estimates  of  ^,  the  effect  of  5  min/day  CAI  instruction,  aver- 
age to 

i  =  [-.04  +  (-.25)  +  (-.06)]/3  =  -.12. 
The  seven  estimates  of  p,  the  effect  of  10  min/day  CAI,  average 
to  \  '  : 

'      p  \  [.23  +  .26  +  .18  +  (-.09)  +  .31  +  .51  +  .03]/7  =  .20. 
The  sfijgle  estimate  of  e,  the  effect  of  20  min/day  of  CAI,  is 

\  e  ='.45. 

This  is  (i^^finitely  a  trend  in  the  expected  direction,  and  the 

distinction  among  number  of  minutes  of  instruction  per  week  accounts 

\ 

for  some  of  the  r^^oublesome  variability  in  the  various  estimates  of 

effects.    For  example,  the  11  estimates  in  Table  6  range  from  -.25  to 

\  ■ 

+  .51  with  a  variance^ of  .06.    But  the  (weighted)  average  variance  of 
^    effects  within  the  5  min/day  and  the  10  min/day  categories  (20  min/ 
day  is,  of  course,  not  considered  since  it  has  only  one  estimate  of 
effect)  is  .04,    Thus,  the  instructional- time  distinction  reduces 
variability  in  estimates  bf  effects  by  40%. 
c)    Summary  for  Reading  Achievement 
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Effect  Sizes  Classified  by 
Reading  Vocabulary  Sf  Compreljension, 
Grade  Level  &  Amount  of  Instruction 


Amount  of 
Instruction 


Reading  Vocab. 
(Grades  4-6) 


Reading  Comprehension 
(Grades  4-6) 


5  min/day 
for  one  year 


.23 


12 


10  min/day 
for  one  year 

20  min/day 
for  one  year 


.23 


.23 


.20 


.45 
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3)    Language  Arts  Achievement. 

The  design  of  experimental  comparisons  for  assessing  the 
effects  of  CAI  language  appears  .in  Table  7.    Beside  the 
designation  of  the  On  and  Off-strands,  there  appears  the 
parameter  estimated  by  the  comparison  (either  A^,  X^q  or 

for  5  min/day,  10  min/day  or  20  min/day  of  language  instruction) . 


Table  7 

Experimental  Comparison  for  CAI 
Language  Instruction 


Year 


Grade 

.    3      .  ML  v.-  MM:  X^q  L  v.  M:  X^q 

4  MRL  V.  MM:  \  RL  v.  MM:  X^q         L  v.  M:  X^q 
RL  V.  MM:  X^^ 

5  MRL  V.  MM:  Xg  RL  v.  MM:  X^q 

RL  V.  MM:  X^q 

6  LL  V.  MM:  X^q  MRL  v.  MM:  Xg 

RL  V.  MM:  X^q       '  RL  v.  MM:  X^q  / 


♦Instruction  in  Year  1  lasted  only  from  February  through  April. 
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a)    Achievement  in  Spelling,  Mechainics  &  Expression/ 

The  estimated  effect  sizes  corresponding  to  the 
comparisons  in  Table  7  appear  in  Table  8.  / 

Table  8 

Effect  Sizes  for  the  Experimental  Comparisons 
in  Table  7  for  Language  Arts  Achievement: 
.  Spel ling/Mechanics/Expression. 


Grade 

1 

Year 
2 

3 

4 

3  X^q:  -.12/. 10/. 01  X-^q:  .51/.31/.12 

4  .    X5:  +.08/-. 01/-. 07  X^q:  .03/. 41/. 33  X■^Q^.  .48/. 44/. 20 

xX^g:  .34/. 21/. 21 

5  \  Xg:  -.16/. 16/-. 25  X^qI  -.09/. 21/. 33 
,    '                          X^q:  .22/. 16/-. 38 

6  X20=  -.10/. 08/-. 09  Xg:  -.13/.07/.17 
X^q: -.11/.18/.12                                 X^q:  -.03/.21/.20 


First,  consider  all  estimates  A^q  and  average  them  to  determine  / 
whether  trends  exist  for  grade  level  or  year: 
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Average  x^q  for  Grades  &  Years,  . 
(Spell ing/Mechanics/Express ion). 


Year 

1 

-i-  _L 

_4_ 

.23/. 20/. 17 

.05/. 13/-. 19     .00/. 31/. 27 
Grade 

.30/ 

.32/. 22 

.._3 

-1-  J_ 

6_ 

.21/. 21/. 07 

.28/. 35/. 25     .07/. 18/-. 02 

.04/. 

20/. 16 

Inspection  of  the  average  effects  across  either  grades  or  years 
shows  no  consistent  trends-    Both  grade  and  year  can  probably  be  safely 
ignored  in  the  analyses  that  follow.    In  the  following  table,  the 
estimated  effects  for  5  min/day,  TO  min/day  and  20  min/day  of  CAI  language 
instruction  are  compared: 

Average  Estimated  Effect  Sizes: 

Spelling       Mechanics         Expression  (#  of  Effects) 
5  mins/day,  X^:          -.07                .07                -.05  (3) 
10  mins/day,  X^^:         .16               .25                  .13  (9)  • 

20  mins/day,  X^q:       --10  .08  -.09  (1) 

The  above  averages  are  indeed  small,  but  a  trend  may  be  lurking 
in  them.    First,  the  three  20  min/day  figures  probably  ought  to  be 
eliminated  from  consideration  since  they  are  based  on  a  single  experimental 
comparison  performed  in  Year  1.    The  5  min/day  averages  look  to  be  genu- 
inely zero.    The  10  min/day  averages  remain.    They  are  small  (.16  for 
spelling,   .25  for  mechanics,  .13  for  expression)  but  they  hint  at  a 
reliable  non-zero  effect.    Consider  the  graph  of  the  effects  in  Figure  2. 
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These  are  the  27  effect  sizes  for  CAI  language  instruction  when  instruc- 
tion was  conducted  for  10  min/day  for  one  school  year. 

The  arrow  in  Figure  2  marks  the  average  of  the  27  effect  sizes. 
However,  one  of  the  27  is  clearly  an  "out! ier viz. ,  the  value  of 
-.38  for  Grade  5  -  Year  2  (cf.,  Table  A-8,  p.  412  in  Dixon  and  Massey, 
1957).    Elimination  of  the  outlier  raises  the  average  to  +.18.  It 
seems  advisable  to  remove  the  aberrant  point  even  though  the  reasons^ 
for  its  uniqueness  are  not  known.    The  26  remaining  effect  sizes  can 
be  averaged  and  arranged  in  a  table  in  which  type  of  language  skill  and 
amount  of  instruction  are  represented. 


-.5  -4   -3  -.2  -.1     0    .1    .2   .3    .4  .5 


Figure  2.  Effect  Sizes  for  CAI  Language  Arts  Instruction, 
b)    Summary  for  Language  Arts. 


Spell ing 

Mechanics 

Expression 

Average 

5  mins/day: 

-.07 

.07 

-.05 

-.02 

10  mins/day: 

.16 

.25 

.19 

.20  , 

Average: 

.04 

.16 

.07 

.09 

These  data  appear  to  indicate  no  added  benefits  for  5  minutes/day  of 
language  arts  CAI  but  consistent  small  effects  for  10  minutes  of  instruction. 
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B.    The  Longitudinal  (Three-Year)  Studies 

In  Table  9  appear  the  estimated  experimental  effects  on  math,  reading 
and  language  achievement  from  the  two  three-year  longitudinal  studies 
using  the  CAI  On-strand  vs.  CAI  Off-strand  design.    The  source  of  the 
data  upon  which  the  estimates  are  based  is  the  report  of  test  scores  in 
the  final  report. 

The  sensitivity  of  standardized  language  arts  tests  to  local 
curricula  has  often  been  questioned;  moreover,  the  success  of  computers 
in  promoting  rote  learning  as  opposed  to  conceptual  learning  has  frequently 
been  suggested.    Both  phenomena  appear  to  be  reflected  in  Table  9. 
There  is  a  solid  benefit  of  between  a  half  and  three  quarters  of  a 
standard  deviation  for  math  computation.    The  addition  to  the  regular 
instructional  program  of  20  minutes  each  day  for  three  years  of  CAI 
teaching  in  math  appears  to  have  added  about  .70  standard  deviation  units 
of  achievement  to  the  average  student's  skill  in  math  computation.  The 
benefits  of  the  CAI  instruction  on  his  knowledge  or  understanding  of 
math  concepts  and  applications  is  less  clear. 

Despite  the  likely  mismatch  between  the  language  subtests  and  the 
languagj^  arts  CAI  curriculum,  there  appear  to  be  small  positive  benefits 
accruing  for  10  min./day  of  language  Instruction.    The  variability  in 
reading  effects  across  the  two  studies  is  so  great  as  to  preclude  any 
final  statement  on  the  benefits  of  the  long  term  use  of  the  reading  prog- 
ram. ^ 

Some  the  the  relationships  between  the  control  and  CAI  groups  are 
depicted  in  Figure  3.  One  sees  that  the  average  Math  CAI  pupil  scores 
about  76  percent  of  the  control  pupils  on  Math  computation  after  three 
years. 
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Table  9 

Effects  of  Additional  CAI  Instruction  in  Basic  Skills  on 
Achievement  In  Math,  Reading  &  Language  for  Three-year  Longitudinal 
Treatment  Study:    CAI  On-strand  vs.  Off-strand  Design. 


CAI  On-strand  of 
20  min/day  vs. 
Achievement        CAI  Off-strand 
Test  (n^  =  n2  =  30) 

Grades  4-6. 


Math: 

Computation 

Concepts 

Appl 1  cations 

Reading: 
Vocabulary 

Comprehension 

Language : 
Spelling- 
Mechanics 
Expression 


A  =  .64 
.28  ^ 
.58 

.58 
-.24 

.22 
.09 
.22 


CAI  On-strand  of 
20  min/day 

(n^  =  n2  =  30) 
Grades  3-5. 

.79 
-.10 
-.05 

-.04 
.24 

.06 
,40 
.24 


Mean  Effect 
Size 

0.7t 
0.09 
0.26 

0.27 
0.00  , 

0.14 
0.25 
0.23 
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\        \        CAI  Groun 

Control 


Math  Comoutation 


76th  ?-ne  of 
Control  GrouD 


10  cr 


CAI  Group. 


Math  Concents 


54th  of 
Control  GrouD 


^"ontrol 


25  or 


60th  «-ne  of 
Control  GrouD 


Reading  Vocabulary 


Fiaure  3.  Effects  of  CAI  on  Math  Computation,  Math  Concents  and  Readinq 
Vocabulary  from  the  Three-year  Longitudinal  Study 
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C.    Summary  of . Effectiveness  Analyses 

T^e  data  analyzed  here  on  the  effects  of  CAI  instruction  on  reading, 
math  and  language  arts  are  summarized  in  this  section.    Findings  are 
not  reported  separately  by  year  of  the  study,  since  this  distinction 
made  no  difference  in  the  size  of  the  effect  prodOced  by  CAI.  Likewise, 
grades  are  lumped  into  Grades  2-3  and  Grades  4-6,  because  the  possibility 
of  an  interaction  of  effectiveness  witK  grade  exists  for  math  achievement'. 
Amount  of  instruction  per  day  (5,  10  or  20  minutes)  was  an  important 
variable  and  is  represented  in  the  table.    See  Table  10. 

Table  10  is  already  a  highly  condensed  table  in  which  details  are 
ignored  (one  hopes  none  of  them  are  very  important)  for  the  sake  of 
coming  to  an  understandable  conclusion.    In  spite  of  the  condensation 
in  Table  10,  one!s  eye  is  still  distracted  by  incidental  features. 
Perhaps  the  findings  can  be  made  clearer  by  an  "enhancement"  of  the  pic- 
ture, in  the  manner  of  computer  enhanced  photographs.    Table  11  is  based 
on  Table  10;  it  was  produced  by  exaggerating  the  "signal"  in  Table  10  and 
damping  the  "noise."    What  is  signal  and  what  is  noise  is  difficult  to 
judge.    The  following  rules  were  used  to  enhance  Table  11:    (a)  negative 
numbers  were  changed  to  zero,  (b)    most  positive  numbers  were  rounded 
to  the  nearest  ,05  number,  (c)    grade-by-treatment  interactions  were 
ignored  (by  averaging  over  grades)  since  they  were  based  on  less  solid 
evidence,  (d)    other  things  equal,  large  sample  results  should  count 
more  heavily  than  small  sample  results. 


Table  1Q 


Effect  Size  (in  Standard  Deviation  Units)  for  Various 

Amounts  of  Computer-Assisted  Instruction  for 
Achievement  in  Mathematics,  Reading  &  Language  Arts 


Math 


Reading 


Language 


Hins./day   Years     Grades;  2-3  4 


10 
20 


1 


Computation     Concepts   Applications    Vocabulary  Comprehension, Spellin^.^  Mechanics  Expression 

/  4-6     2-3     4-6     2-3     4-6     2-3     4-6     2-3     4-6  ^-3     4-6     2-3     4-6  .. 

.23  -.12  -.07  \ 


.27  .26  -.07  -.07  .07  .07 
.63    .37    -.02    -.02      .04   "  .04 


.23 
.23 


.20 

.45 


.16 


-.05 
.19 


10 

20: 


3 

'  3 


71 


,09 


•.26 


(Where  cells  are  blank,  there  were  no  data.) 


■  .27 


.00 


,14 


,25 


,23 


00 


Table  11 
"Enhanced"  Version  of  Table  lo 

Language 
Spell.       Mechan.  Express. 

.15  .25  .20 


10  min.  3 

20  min.        3  .70 


*                    Math  Comp. 

Grades  Grades 

Mins/day        2-3  4-6  - 

10  min.        .30  .25 

20  min.       .65  .35. 


-  Math  ^  Reading 

MT"s/day      Years  Comput.*  Concept      Appl.  Vocab.  Compre. 

1                 .25  .00  '     .00  .25  .20 

20  min.         1                 .50  .qO  .00  .25  -45 


,25 


.00 


.15 


.25 


.25 


.10 


,25 


4S8 


id 

ERIC 
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III.    THE  EFFECTS  OF  REPLACING 

TRADITIONAL  INSTRUCTION  \ 
BY  COMPUTER  ASSISTED  INSTRUCTION:  \ 
FINDINGS  FROM  THE  COHORT  CONTROL  DESIGN  V 

\ 

The  comparison  of  the  achievement  of  pupils  studying  by  CAI  with 
the  achievement  of  pupils  who  were  taught  one  year  earlier  or  later  in 

the  same  school  is  called  a  Cohort  Control  Design,  and  it  addr^esses  a 

• . .  b  .  ^, 

different  question  from  the  findings  of  the  Within  -  CAI  Designs 
Since  total  instructional  time  was  equated  between  the  CAI  groups  and 
the  Cohort  control  groups,  the  CAI  vs.  Cohort  comparisons  measure  the 
effect  of  replacing  some  time  (10  or  20  mins.  two  to  five  times^  a  week) 
in  traditional  instruction  by  the  same  amount  of  time  with  CAI.  (The\ 
findings  reported  earlier  measured  the  effect  of  adding  CAI  instructiori\ 
to  the-existing  curriculum,  which  was  otherwise  unchanged.)  \ 

The  Cohort  design  was  earlier  judged  inferior  to  the  Within  -  CAI  desi\gn; 
to  repeat  what  was  written  about  the  val.idity  of  the  Cohort  design   earlier  \ 
in  this  report,  "The  validity  of  the  CAI  vs.  Cohort  Control  design  is  \ 

even  more  problematical  than  that,  of  the  Within  -  CAI  Control  design,  for  the  ^ 
reason  that  assignment  between  CAI  and  Cohort  Coritrol  groups  was  non-random 
in  unknown  ways.    Designation  of  CAI  vs.  control  years  for  Cohort  control  was 
somewhat  arbitrary  and  might  be  regarded  'as  if*  random;  but  even  so,  it  was 
as  if  random' at  the  school  level  at  best  and  not  at  the  level  of  pupils." 

The  validity  of  the  Cohort  design  is  probably  not  as  bad  as  was  earlier  . 
indicated.    A  feature  of  the  CAI  schedule  permits  estimation  of  and  correction 
for  most  of  the  weaknesses  of  the  Cohort  design.    Within  a  CAI  classroojn,  pupils 
were  randomly  ^assigned  to  CAI  curriculum  groups,  such  a§-M,  R,  1,  RL  and  so 
forth.    When  al  CAI  M-group  is  compared  with  a  Cohort  control  group  from  the 
same  school  but  from  an  earlier  (non-CAI)  year,  the  comparison  is  confounded 


by  a  large  number  of  unknown  non-random  influences  plus  it  is  presumed  to 
benefit  unfairly  from  the  "novelty"  effect  of  CAI  (i.e.,  pupils  may  attend 
to  instruction  more  closely  only  because  it  is  incidentally  associated  with 
the  excitement  of  computers).    However,  the  CAI  R-group,  which  studies 
Reading  but  neither  math  nor  language  by  CAI,  is  subject  to  both  confounding 
influences  (non-random  nonequi valence  and  novelty)  to  the  same  degree  as  the 
CAI  M-group.    Consequently,  in  assessing  the  impact  of  CAI  on  marth  achieve- ' 
ment,  the  CAI  R-group's  math  scores  provide  an  excellent  correction  for. the 
confounding  influences  affecting  the  CAI  M-group  and  Cohort  control  comparison.' 
Hence,  where  possible  in  the  analyses  that  follow,  a  CAI  vs.  Cohort  comparison 
will  be  corrected  by  subtracting  the  "effect"  of  CAI  for  the  comparison  of 

an  equivalent  group  studying  a  different  curriculum  by  CAI  and  its  Cohort 

^  ft 

control.  The  validity  of  th^ resulting  corrected  effect  should  be  quite  respect 
able. 

In  Tables  12-14  appear  the  estimated  effect  sizes  arranged  by  subject 
tested  (mathematics,  language  and  reading),  the  year  of  the  four-year  ETS  study 
in  which  effects  were  assessed  (either  year  #3  or.  #4),  the  duration  of  the  CAI 
instruction  with  the  treatment  group  (1,  2  or  3  school  years^),  and  the  school 
grade  level  (2-5). 

A.    Mathematics  Achievement. 

In  Table  12  appear  the  findings  for  mathematics,  reported  on  three  subtests 

(computation,  concepts  and  applications,  the  latter  two  subtests  being  combined 

at  grades  1  and  2).    All  of  the  effect  sizes  at  grades  4  and  5  are  controlled, 

in  the  sense  that  there  is  a  within-CAI  randomly  equivalent  curriculum  group 

(either  RL,  R  or  L)  that  is  compared  to  the  same  Cohort  control.    The  average 

effect  sizes  for  the  five  experimentarl  comparisons  at  grades  4  and  5  before- 

correction  by  subtracting  the  estimated  effects  of  invalidity  from  the  RL,  R 

or  L  vs.  Cohort  comparisons  are  as  follows: 

i7l 


jtrcatTon/  tonceptsMpP  1 1  can  ons , 
Year  of  the  Study 


^^^^^^ 


sign 


Duration  of  CAI  Instruction 


Grade  of  Achiev. 
Meas . :  . 


MM  V.  Coh. 
(.77/. 29/. 36) 


MM  V.  Coh. 
(.70/. 18/. 28) 


RL  V.  Coh.** 

f  ■17/./15/.5U 


RL  V.  Coh 
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'    Uncorrected  Average 
Effect,  Grades  4  &  5 

Computations  .69 

Concepts  .35 

Applications  .43 

When  these  figures  are  corrected  by  subtracting  the  RL,  R  or  L  vs. 

Cohort  effects,  the  following  averages  are  obtained: 

Corrected  Average  Effect 
Sizes,  Grades  4  &  5 

Computations  .49 

Concepts  .08 

Applications  .08 


These  figures  agree  reasonably  well  with  the  estimated  effects  in  Tables 
10  and  11  for  20  mins/day  of  CAI  for  instruction  of  duration  1  to  3  years, 
the  largest  discrepancy  beina  a  larger  effect  on  Applications  in  the  previous 
analyses. 

The  estimates  of  effects  at  grades  1-3  in  Table  12  are  uncontrolled  in 

the  sense  that  they  lack  within-CAI  curriculum  groups  (RL,  R  or  L)  vs.  Cohort 

f 

control  comparisons  for  correcting  for  nonequivalence  and  novelty  effects. 
These  estimates  of  effects  differ  further  from  those  at  the  higher  grades  in 
that  the  Concepts  and  Applications  subtests  are  merged.    The  average  effect 
sizes  on  the  subtests  are.  as  follows: 

Uncorrected  Average 
Effect  Sizes,  Grades  1-3 

Computations  .41 

Concepts  &  Applications  .13 
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These  figures  show  the  same  trends  as  -the  comparable  figures  in  Table  10 
and  agree  reasonably  wel 1 ,  though  not  precisely,  with  them. 

B.  Reading  Achievement. 

In  Table  13  appear  the  estimated  effect  sizes  for  the  Reading  subtests  of 

Vocabulary  and  Comprehension.    The  M  and  L  CAI  group  vs.  Cohort  control  effect 

sizes  can  be  used  to  correst  the  R  and  RL  vs.  Cohort  control  effects.    Each  of 

the  five  CAI  Reading  effect  sizes  in  Table  14  can  be  corrected  in  this  manner; 

the  R  vs.  Cohort  effect  for  grade  4  -  year  4  was  corrected  by  the  average  of 

the  effects  for  M  vs.  Cohort  and  L  vs.  Cohort.    The  average  corrected  effect 

sizes  for  the  Reading  subtests  are  as  follows:  ^ 

Reading  Corrected  Average 

Subtest  Effect  Size  ' 

Vocabulary  .14 

Comprehension  .15 

These  effects  are  very  nearly  zero. 

C.  Language  Arts  Achievement 

In  Table  14,  the  estimated  effects  appear  for  the.  Language  subtests  of 

Spelling,  Mechanics  and  Expression.    With  respect  to  the  assessment  of  language 

outcomes^  the  M  and  R  CAI  vs.  Cohort  control  effects  can  be  used  to  correct  the 

RL  or  L  vs.  Cohort  control  effects  for  the  nonequi valence  of  the  two  latter 

groups  and  the  novelty  advantage  of  the  CAI  groups  over  the  Cohort  control  groups 

Unlike  in  Table  12,  all  of  the  CAI  language  effects  can  be  corrected.  Their 

average  for  each  subtest  is  as  follows: 

Language  Corrected  Average 

Subtest  Effect  Size 

Spelling  -.03 

Mechanics         .  .25 

"  Expression  .15 


n 


Table  13:    Effect  Sizes  for  Reading:    (Vocabulary/Comprehension)  from  CAI  vs.  Cohort  Design 

Year  of  Study 

3  4 

Duration  of  CAI  Instruction 


2  yrs.  2  yrs.  3  yrs 


Grade 


4 


5 


RL(3)  V.  Coh. 
(.n/.34) 

R  V.  Coh. 
(.35/. 25) 

MM(3)  V.  Coh.** 
(.18/. 29) 

RL(2)  V.  Coh. 
(.59/. 68) 

MM(2)  V.  Coh.** 
(.22/. 34) 

M  V.  Coh.** 
(-.05/-. 02) 
L  V.  Coh.** 
(.25/. 16) 

RL  V.  Coh. 
(.50/. 44) 

RL  V.  Coh. 
(.41/. 20) 

MM  V.  Coh.** 
(.30/. 27) 

MM  V.  Coh.** 
(.34/. 18) 

**These  effect  isizes  estimate  the  combined  effects  of  experimental  internal  invalidity  and 
CAl  novelty. 

cn 
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Table  14:    Effect  Sizes  for  Language.:  (Spelling/Mechanics/Expression)  from  CAI  vs.  Cohort  Design 

Year  of  Study 

3  4 
Duration  of  CAI  Instruction 


2  yrs.  2  yrs.  3  yrs. 


Grade 


5 


RL(3)  V.  Coh. 
(-.16/. 66/. 56) 

L  vs.  Coh 
(.07/. 60/. 38) 

MM(3)  V.  Coh.** 
(-.01/. 20/. 43) 

RL(2)  V.  Coh. 
(.03/. 94/. 92) 

MM(2)  V.  Coh.** 
(.09/. 47/. 31) 

R  V.  Coh.** 
(.00/. 54/. 52) 
M  V.  Coh.** 
(-.24/. 36/. 34) 

RL  V.  Coh. 
(.21/. 57/. 34) 

RL  V.  Coh. 
(.20/. 49/. 18) 

MM  V.  Coh.** 
(.33/. 43/. 29) 

MM  V.  Coh.** 
(.20/. 47/. 16) 

**These  effect  sizes  estimate  the  combined  effects  of  experimental  internal  invalidity  and 
CAI  novelty. 
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These  agree  generally  with  the  effect  sizes  reported  in  Table  10  for 
the  Language  subtests  primarily  in  their  being  quite  small. 

D.    Summary  of  Effects  of-CAI  Replacing  Traditional  Instruction 

The  findings  in  Tables  12-14  may  have  been  too  incomplete  (in  a  factorial 
design  sense)  to  detect  trends  in  effect-size  magnitude  across  grades  or  years 
of  the  study  or  duration  of  instruction  and  separate  these  possible  trends  from 
interactions.    Where  trends  could  be  explored,  none  emerged,  e.g.,  in  Mathe- 
matics at  grades  3  and  5  where  duration  of  instruction  trends  might  have  shown 
up.    Hence,  in  the  following  summary  of  effect  sizes,  distinctions  among  years 
of  the  ETS  study,  school  grades  and  duration  of  instruction  in  years  have  been 
dropped.    The  figures  in  Table  15  have  been  slightly  "enhanced"  through  averag- 
ing (e.g..  Grades  4  &  5  with  Grades  1-3  in  math)  and  rounding  sp  as  to  disspell 
the  impression  of  accuracy. 
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Table  15:    Corrected  Effect  Sizes      Estimated  from  the 
CAI  vs-  Cohort  Control  Design 


Corrected 

Subject  Tested'  Effect  Size 

Mathematics  . 

Computations  .45 

Concepts  .10 

Applications  .10 


Reading 

Vocabulary  .15 
Comprehension  . 15 


Language 

Spelling  '.00 

Mechanics  .25 

Expression  .15 


473 


49 


IV.    NOTES  ON  ALTERNATIVE  METHODS 
OF  IMPROVING  ACHIEVEMENT 

The  following  alternatives  to  CAI  as  a  meaBS  of  increasing  pupil 
achievement  will  be  considered:    (1)  reduction  in  class  size,  (2) 
tutoring,  (3)  instructional  television,  (4)  teacher  training,  (5) 
calculators. 

The  instructional  method  of  choice  will  be*  that  one  which  is  most 
cost-effective.    In  the  pages  that  follow,  much  is  reported  about  the 
effectiveness  in  terms  of  pupil  achievement  of  several  alternatives  to 
CAI.    However,  these  few  notes  fall  far  short  of  a  defensible  cost- 
effectiveness  analysis  since  no  attempt  was  made  to  evaluate  costs. 
While  the  following  data  on  the  effectiveness  of  various  methods  of 
instruction  may  help  plac6  the  CAI  effectiveness  data  in  a  slightly 
more  informative  context,  closure  on  the  question  must  await  a  more 
serious  attempt  at  cost-effectiveness  analysis,  (of  the  type,  for 
example,  that  Jamison  (1982)  performed  on  class  size  reduction  vs.  CAI). 
1 .    Reduction  in  Class  Size. 

School  classes,  which  typically  number  about  25  pupils  today  in 
the  elementary  grades,  can  be  made  smaller  by  hiring  more  teachers,  by 
putting  aides  (paraprofessionals)  in  the  classroom,  or  by  using  older 
pupils  to  teach  or  tutor  younger  ones.    Research  has  shown  that  inter- 
ventions of  any  of  these  types  results  in  improved  pupil  achievement, 
and  that  no  one  of  these  interventions  has  been  shown  to  be  any  more 
effective  than  the  others  (Glass,  Cahen,  Smith  and  Filby,  1982). 

In  Figure  4  appears  a  graph  of  the  relationship  jDetween  school 
class-size  and  pupil  achievement.    The  relationship  was  empirically  deter- 
mined by  the  findings  of  over  700  comparisons  of  achievement  in  classes 
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of  varying  sizes  from  nearly  80  studies  published  since  1910.    The  actual 
shape  of  the  curve  is  based  on  a  subset  of  110  comparisons  taken  from  14 
studies  that  applied  good  experimental  principles  in  the  design  of  the 

comparison  of  smaller  and  larger  classes.   

The  curve  in  Figure  4  is  interpreted  as  follows:    if  a  class  of 
size  25  is  assumed  to  have  average  achievement  (z  =  0),  reduction  of 
class-size  by  5  pupils  will  raise  achievement  by  about  .05  standard 
deviation  units,  reduction  to  class-.size  15  will  increase  achievement 
by  about  .15  standard  deviation  units,  and  so  on. 

Several  points  must  be  noted  about  these  findings  because  they 
bear  on  the  comparison  of  these  data  with  the  ETS-LA  findings  for  CAI: 

.     a)  no  evidence  was  found  that  the  basic  relationship  in  Figured 
was  different  for  different  subjects  taught  (e.g.,  reading, 
math),  different  grades  (elementary  vs.  secondary)  or  for 
"teachers"  of  different  levels  of  experience, 
b)  all  of  the  data  underlying  Figure  4  were  based  on  experiments 
in  which  small  classes  replaced  rather  than  supplemented 
large  classes,  i.e.,  the  small  class  was  taught  for  the 
same  amount  of  time  as  the  large  class,  as  opposed  to  a 
design  in  which  pupi'is  normally  in  large  classes  were 
given  additional  instruction  in  small  groups.    Thus,  the 
findings  in  Figure  4  for  reduction  in  class-size  are  more 
comparable  to  the  ETS-LA  cohort  comparisons  than  to  the 
Within-CAI  comparisons. 


Figure  4.    Graph  of  the  Relationship  Between  Instructional 
Group  Size  and  Achievement. 
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c)  The  duration  (in  hours  and  weeks)  of  the  experiments  that 
gave  rise  to  the  data  underlying  Figure 4- varied  greatly, 
from  as  little  as  1  hour  .to  as  many  as  900.    Even  though 
no  relationship  was  observed  between  the  duration  of  the 
study  and  the  strength  of  the  relationship  between  class- 
size  and  achievement,  I  have  tabulated  the  study  durations 
below: 

Hrs  Instruction         No.  of  A's 


1 

9 

3 

1 

9 

3 

25 

8 

30 

18 

40 

1 

54 

2 

60 

1 

100 

5 

120 

19 

180 

2 

700 

36 

900 

3 

Just  what  is  to  be  made  of  this  failure  to  discern  a 
relationship  between  the  number  of  hours  in  small  group 
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instruction  and  the  size  of  the  benefit  of  small  classes 
is  troubling,  particularly  for  cost-benefit  analysis. 
Obviously,  the  no  relationship  finding  can  not  be  extra- 
polated to  the  limit  or  else  one  would  be  forced  to  conclude 
that  one  minute  of  instruction  in  a  one-on-one  tutorial 
is  the  equivalent  in  benefit  (.87  standard  deviation  units) 

of  a  whole  school  year  of  tutorials.    Clearly  this  can 
not  be  so. 


'I  Si 
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2.  Tutoring. 

"Tutoring"  (i.e.,  one-on-one  instruction)  can  be  regarded  as  a  special 
case  —  the  limit  —  of  class-size  reduction.    Studies  of  its  effectiveness 
in  relation  to  instruction  in  groups  were  covered  in  the  previous  section 
on  the  effects  of  class-size  reduction.    If  the  class-size  and  achievement 
curve  in  Figure  1  is  to  betelieved,  a  pupil  who  is  tutored  individually  scores 
.87  standard  deviation  units  higher  on  an  achievement  test  than  he  would  if 
taught  in  a  group  with  24  other  pupils. 

Hartley  (1977)  investigated  the  benefits  of  tutoring  in  a  meta- 
analysis of  the  effects  on  mathematics  achievement  of  methods  of  individual- 
izing instruction.    She  identified  68  effect  size  measures  from  about  25 
studies  of  the  effects  of  supplemental  tutoring  (i.e.,  in  addition  to  large 
group  traditional  classroom  instruction).    Only  5  effect  size  estimates  were 
available  for  replacement  tutoring  (i.e.,  tutoring  in  place  of  traditional  large 
group  instruction),    ^he  average  and  the  standard  deviation  of  the  effects  were 
as  follows:  | 

!        Effect  Sizes  for  Tutoring 

* 

St.  dev. 

Average  of  effect      No.  of 

Type  of  Tutoring        Effect  Size      "      s i zes         Effect  Sizes 


*  Suppleme|ital  .61  .70  68 

Replacemfent  .42  .26  5 

The  "Replacemint"  findings  should  not  be  taken  too  seriously  since  they 
.     are  based  on  so  few  studies.    The  "Supplemental"  average  effect  is  not  directly 
comparable  to  the  tutoring  estimate  of  .87  from  the  class-size  analysis  since 
supplemental  studies  were  excluded  from  the  class-size  study  by  the  nature 
of  the  question  addressed  there. 
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Hartley  reported  the  following  data  on  the  amount  of  tutoring  given 
pupils  In  these  studies:  the  average  number  of  hours  of  instruction  was  37.4 
hrs.  (with  a  standard  deviation  of  43.9  hrs.)       hence,  they  were  more  hetero- 
geneous in  terms  of  hours  of  instruction  than  they  were  long!    The  average 
length  of  t\}e  experiments  in  weeks  was  about  one  semester:  17.0  weeks 
(standard  deviation  =  12.2  weeks).    Moreover,  the  correlation  between  the 
duration  of  application  of  the  experimental  treatment  and  the  effect  s^ze 
was  £  =  -.01 • 

Hartley  also  collected  data  on  computer-assisted  instruction  in  mathe- 
matics.   Where  CAI  supplencnted  regular  large-group  instruction,  the  following 
comparison  of  CAI  and  tutoring  was  observed: 

Effect  Sizes  by  Type  of  Supplement 


CAI   Tutoring 

Average  Effect  .47  .61 

St.  dev.  of  Effects  .58  .70 

No.  of  Effect  Sizes  75  68 


Here  the  superiority  of  Tutoring  over  CAI  is  of  the  order  of  about  .15 
•standard  deviation  units.    If  one  restricts  attention  to  experiments  in  which 
pupils  were  randomly  assigned  to  experimental  and  control  groups,  the  Tutoring 
advantage  widens  and  the  typical  CAI  effect  becomes  much  more  like  that  ob- 
served in  the  ETS-LA  study: 

Effect  Sizes  from  Randomized 
Designs  by  Type  of  Supplement 

CAI  Tutoring   

Average  Effect  31  5g 

St.  dev.  of  Effects          ,44  ^74 

No.  of  Effect  Sizes  n  52 
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The  above  figures  seem  credible.    They  show  an  effect  of  about  .30 
standard  deviation  units  for  supplemental  CAI  and  about  .60  for  supplemental 
tutoring.    It  should  be  pointed  out  that  the  average  number  of  hours  of 
instruction  in  the  CAI  studies  summarized  to  obtain  the  .30  effect  size 
was  22.1  hrs.  vs.  37.4  hrs.  for  the  tutoring  study.    If  the  evidence  of 
Hartley's  analysis  on  the  relationship  of  duration  of  instruction  and 
benefits  can  be  set  aside  momentarily  and  the  assumption  made  that  amount  of 
supplemental  instruction  should  be  linearly  related  to  effect  size,  then  the 
.30  effect  of  CAI  could  be  inflated  by  the  factor  37.4/22.1  =1.69  to  obtain  a 
CAI  effect  for  a  period  of  instruction  comparable  in  length  to, that  for  tutor- 
ing; CAI  for  37.4  hrs.  would  equal  1.69  (.30)  =  .51  standard  deviation  units;. 
Now  the  comparison  of  CAI  and'  Tutoring  effect  sizes  looks  more  like  .Sl  'vs. 
.53,  a  smaller  difference. 

There  were  other  relationships  in  Hartley's  data  that  argue  that  CAI 
and  Tutoring  are  even  more  similar  in  their  impact  on  achievement  than  they 
first  appear  to  be.    The  CAI  effect  is  slightly  negatively  correlated  with 
grade  level  and  negatively  correlated  with  pupil  IQ.    Tutoring  effects  are 
less  strongly  negatively  correlated  with  grade  level,  and  they  are  positively 
correlated  with  pupil  IQ.    When  the  effect  sizes  for  CAI  and  Tutoring  are 
covariance  adjusted  and  estimated  at  grade  6  for  low  ability  pupils,  (Hartley, 
1977,  p.  81)  the  effect  sizes  are 

CAI  =  .431, 
Tutoring  =  .427. 

Several  different  figures  have  been  given  above;  they  are  repeated 
here  for  clarification: 

'1^7 
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Average  Effect  Sizes 


CAI 


Tutoring 


1.  Replacement  of 
traditional  classroom 
instruction. 

2.  Supplement  of 
truditional  classroom 
instruction. 

3.  Supplemental  & 
randomized  design. 

4*    Same  as  #3  but 

instructional  time 
equated.  . 

5*    Same  as  #3  but  with 
IQ  and  grade  level 
.    equated,  (Low  IQ, 
Grade  6)- 


.47 


.31 
.51 


.43 


,87  (from  Glass  et  al_. ,  1982) 
.42  (from  Hartley,  1977) 

.61 

.58 
.58 

.43 


Precisely  what  is  to  be  made  of  these  different  figures  is  hardly  obvious 
The  replacement  efifect  size  for  tutoring  might  be  somewhere  between  .40  and  .90! 
For  some  reason  . I  think  a  figure  nearer  .40  is  more  defensible  ihon  one  near 
.90.    Therefore,  the  effect  size  for  Tutoring  as  a  replacement  of  traditional 
large  group  instruction  in_math_js  >.50  standard  deviation  units. 

The  figures  for  the  effect  of  tutoring  used  as  an  add-on  supplement  to 
traditional  large  group  classroom  instruction  agree  more  closely  among  them- 
selves than  the  two  replacement  estimates  did:  the  effect  sizes  ranged  from 
.43  to  .61.    The  average  is  a  good  summary:  therefore,  the  effect  size  for 
^.utoring  as  a  supplement  to  traditional  large  ^roup  instruction  in  math  is  ^.50 
standard  deviation  units. 

Many  estimates  of  CAI  effects  in  math  are  of  the  order  of  .50  standard 
deviation  units.    Considering  line  5  of  the  above  table  and  a  lot  of  other 
things,  my  guess  is  that  the  benefits  for  math  of  equal  amounts  (in  minutes) 
of  CAI  and  tutoring  are  nearly  equal. 
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3'    Instructional  TeleviS'ion. 

Chu  and  Schramm  (1967)  reviewed  207  experiments  comparing  instructional 
television  (ITV)  with  traditional  classroom  teaching;  they  could  see  no  clear 
trend  in  these  studies  indicating  that  pupils  learned  more  with  ITV  than  by 
traditional  teaching  methods. 

Carnoy  (1975)  attempted  a  cost-effectiveness  analysis  of  ITV,  but  it 
is  difficult  to  infer  from  his  report  whether  or  not  he  believes  ITV  is  more  - 
effective  than  traditional  classroom  instruction.    The  Hagers town.  Maryland  ' 
data  reported  in  Chu  and  Schramm  are  cite.d  and  analysed  by  Carnoy,*  but  his 
attitude  toward  them  is  ambivalent.    So  is  mine,  but  they  indicated  the  following; 

Hagerstown:  Effects  of  ITV  on  Mathematics 
Achievement  (Iowa  Test  of  Basic  Skills)  for 
Primary  School  Pupils  (Data  in  G.  E.  Units) 


Year 

Grade  3 

Grade  4 

Grade  5 

Grade  5 

1958  (Before  ITV) 

3.59 

4.43 

5.26 

6.49 

1959  (Yr  1  of  ITV) 

4.06 

4.97 

5.77 

6.83 

1960  (Yr  2  of  ITV) 

4.18 

5.01 

6.13 

7.17 

1961  (Yr  3  of  ITV) 

4.30 

5.08 

6.19 

7.28  • 

For  the  calculation  of  effect-size  measures  from  the  above  data  we  can 

4 

safely  assume  a  standard  deviation  of  1.0  years  for  elementary  school  pupils 
in  math;  this  is  the  figure  typically  Observed. 


*Carnoy  also  reported  ITV  effectiveness  data  from  El  Salvador;  information  needed 
to  translate  the  findings  into  standardized  effect  sizes. was  missing.    I  haven't 
been  able  to  use  the  data. 
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The  above  data  can  be  treated  as  a  cohort  design  with  Grade  3  in 
1958  being  essentially  the  same  pupils  as  in  Grade  4  in  1959  and  so  forth. 
The  1958  (Before  ITV)  averages  can  serve  as  control  group  expectations.  For 
example,  comparison  of  Grade  3-1959  against  Grade  3-1958  is  an  evaluation  of  the 
effect  of  one  ye^r  of  ITV:  (4.06  -  3.59)/1.0  =  +.47  standard  deviation  uni~s. 
Likewise,  the  comparison  of  Grade  4  -  1959  with  Grade  .4  -  1958  gives  aniesty?)ate 
of  the  benefits  of  a  year's  ITV:  (4.97  -  4.43)/1.0  =  +.53  standard  deviation 
units.    There  are  a  total  of  six  such  one-year  effect  sizes.    Similarly,  there 
are  four  comparisci^ns  that  assess  the  effect  size  from  two  years  study  under 
ITV,  e.g.,  Grade '4  -  1950  vs.  Grade  4.-  1958  equal  to  (5.01  -4. 43)/l  .0  =  +.  58 
St.  dev.  units. 

^       In  the  following  table,  the  effect  sizes  reflecting  the  benefits  of 
ITV  as  a  replacement  of  traditional  classroom  instruction  are  reported;  the 
reported  figures  are  averages  of  the  multiple  experimental  comparisons  corres- 
ponding to  one,  two  and  three  years  exposure  to  ITV. 

No.  of  Years  Average 
Under  ITV.  Effect  Size:  Math 

One  +.53 

Two  +.70 

Three  +.86 


Carnoy  estimated  that  these  benefits  were  bought  at  a  cost  of  $31  per 
pupil  per  year. 
4.    Teacher  Training. 

Gage  and  Giaconia  (1981)  reviewed  the  findings  of  four  experiments  on  the 
effects  of  teacher  training  on  pupil  achievement.    They  reported    that  i>pecialized 
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teacher  training  progr^ams  (training  in  pedagogic  methods  dealing  mainly  with 
management  of  large-group  instruction)  produced  achievement  gains  in  experimental*' 
groups  that  exceed  control  group  gains  by  more  than  .5  standard  deviations. 
I  have  been  able  to  obtain  only  two  of  the  four  reports:  Good  and  Grouv/s  (1979) 
and  Anderson,  Evertson  and  Brophy  (1979). 

My  calculations-confirm  an  effect  size  of  about  +.5  on  SRA  mathematics 

.  subtest  in  the  Good  and  Grouv/s  experiment  favoring  the  pupils  of  the  group  of 
teachers-^v^ho  received  special  training  (two  90  min  lectures  and  reading  of  a 
45-page  manual).    pRaVBn't  detected  major  flaws  in  this  study.    The  difference 
in  pupil  performance  (on  the  Metropolitan  Reading  Test)  for  specially  trained 
vs.  control  teachers  in  the  Anderson,  Evertson  and  Brophy  study- is  of  the  order 
of  +.90  standard  deviations  favoring  the  pupil  of  the  specially  trained  teachers. 
This  study  was  not  ats  well  done  as  the  Good  and  Groiiws-  study.    Treatment  and 
control  conditions  were  not  mixed  within  schools;  hence,  the  experimental  design 
employed  the  equivalent  of  assigning  schools  (not  teachers)  to  the  two  conditions 
compared.    Nine  schools  and  27  teachers  participated.    No  indication  was  given 
of  how  schools  were  assigned  to  the  experimental  conditions;  one  doubts  seriously 
that  the  assignment  was  random  --  researchers  seldom  omit  mention  of  random 
assignment  when  they  have  been  lucky  enough  to  bring  it  off.    The  investigators 
apparently  hoped  to  compensate  for  the  deficiencies  of  design  by  administer^'ng 

'^ai)retest  (Metropolitan  Readiness  Test)  to  pupils  and  correcting  (via  analysis 
of  covariance)  the  posttest  reading  average  for  any  lack  of  pretest  equivalence. 
The  pretest  data  showed  the  following: 

Mean         St.  dev.  n 
Exper.  Group         68.98  6*  17 

Control  Group        65.74         7.55  10 

*Rough  estimate 

<> 

 :   •   J 
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These  data  show  a  +.48  standard  deviation  difference  on  the  pretest 
between -the  experimental  and  control  pupils,  favoring  the  former.    Analysis  of 
covariance  never  completely corrects  for  such  non-equi  valence  (Cronbach  et  a^. , 
1977);  plus  it  xan  only  correct  for  measured  nonequi valence  and  in  this  instance 
we  have  reason  to  worry  that  some  important  non-equivalences  between  experimental 
and  control  groups  were  not  measured. 


5.    Electronic  Calculators. 

At  least  for  math  teaching  (but  perhaps  for  other  subjects  as  well), 
hand-held  electronic  calculators  (EC)  may  be  cheapter  and  as  effective  as  CAI. 
Many  experiments  on  the  effectiveness  of  EC  in  math  instruction  have  been  per- 
formed.   Roberts  (1980)  reviewed  most  of  them  and  concluded  that  EC  instruction 
is  generolly  beneficial  as  a  supplement  or  superior  to  traditional  classroom 
instruction  when  tested  as  a  replacement'.    More  than  80%  of  the  literature  is 
re-sorted  in  dissertations  not  otherwise  available.    I  have  been  able  to  obtain 
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reports  of  five  EC  experiments.  Their  findings  are  summarized  in  the  foliowing 
table: 


Results  of  Experiments  on  the 
Effectiveness  of  Instruction  by 

Electronic  Calculator  (vs. 
Traditional  Classroom  Instruction) 


Study 


Hrs. 
Instruction 


Gaslin  (1975) 


Hohlfeld  (1974) 


Schnur  &  Lang 
(1976) 


Schafer,  Bell 
&  Crown  (1975) 

Creswell  &  Vaughn 
(1979) 


50 


2 
40 


Grade 


5 
9 


Effect  Size 


Design 
Quality 


Type  of 
Test 


TC 


5( 


con, 


Good 


Good 


Probably 
Good 


Probably^ 
good 


*Electronic  Calculator  group  allowed  to  use 
calculators  on  the  posttest! 


Curriculum 

specific; 

"Fractions" 

-.36 

"Frac."- 

Retiontion 

.00 

Transfer  test  ^ 

+  .08 

Curriculum 

speci  fic: 

multiplication. 

+.25 

Standardized 

test  of 

computation. 

+  .92 

ETS  math 

+  .21* 

comp.  test 

Curriculum 

specific: 

decimals  & 

percents 

1.34* 

The  findings  of  these  five  studies  are  inconsistent.    Effect  sizes  range 
from  -.36  to  +1.34;  they  average  +.35,  but  there  1s  no  good  reason  to  average 
them.    There  is  no  pattern  of  effects  consistent  enough  to  use  as  a.  basis 
for  comparison  with  other  interventions. 
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6.  Summary 

The  data  on  the  effectiveness  of  the  various  methods  of  instruc- 
tion discussed  in  this  section  may  help  to  put  the  CAI  data  in 
perspective.    Overall,  several  of  the  different  intervention  strategies 
appear  to  prodidce  results  superior  to  traditional  classroom  instruction 

/  ■■ 

A  final  assessment  of  the  cost  effectiveness  of  each  strategy  awaits 
the  further  work  of  economists.  ' 
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"An  Evaluation  of  the  Costs  of  Computer-Assisted  Instruction" 

Abstract 

The  purpose  of  this  paper  is  to  estimate  the  costs  and  cost-feasibility 
of  utilizing  computer-assisted  instruction  (CAT)  for  compensatory  education. 
Cost  data  were  collected  from  an  experiment  on  the  effectiveness  of  CAT  that 
had  been  established  in  Los  Angeles  and  sponsored  by  the  National  Institute 
of  Education.     Based  upon  the  resource  ingredients  approach  to  measuring 
costs,  it  was  found  that  up  to  three  daily  ten  minute  sessions  of  drill  and 
practice  could  be  provided  for  each  disadvantaged  child  within  the  present 
allocation  of  funds  from  Title  I  of  the  Elementary  and  Secondary  Education 
Act  of  1965.     If  the  computer  system  were  shared  between  two  schools,  the 
higher  costs  would  permit  only  two  daily  sessions. 

Costs  weri  also  estimated  for  a  more  advanced  CM  system,  and  somewhat 
surprisingly  the  costs  were  in  the  same  range.    This  finding  reflects  the 
very  heavy  costs  of  "software"  which  do  not  seem  to  decline  with  more 
advanced  technologies.    Also,  it  is  possible  that  the  latter  technology 
will  be  found  to  be  more  effective  at  the  same  cost  level.     However,  since 
cbmparative  effectiveness  data  between  the  CAI  approach  and  other  instruc- 
tional strategies  are  not  readily  available,  such  cost-effectiveness  compari- 
sons  will  have  to  be  deferred  until  some  future  date. 
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AN  EVALUATION  OF  THE  COSTS  OF  COMPUTER-ASSISTED  INSTRUCTION 


INTRODUCTION 

Various  educational  technologies  such  as  educational  radio,  educa- 
tional television,  and  computer-assisted  instruction  have  been  proposed 
in  recent  years  as  partial  solutions  to  both  the  problems  of  rising 
educational  costs  and  the  failure  of  the  educational  system  to  impart 
basic  skills  to  disadvantaged  youngs ters,^^.  The  logic  of  the  cost-saving 
aspects  of  educational  technologies  is  conditioned  heavily  upon  the 
assumption    that  the  high  labor  costs  of  education  can  be  reduced  by 
substituting  relatively  lower-cost  capital  inputs  without  sacrificing  ^ 
educational  results.     The  view  that  certain  educational  technologies  can 
improve  the  quality  of  educational  results  for  disadvantaged  youngsters 
is  premised  on  the  fact  that  such  approaches  as  computer-assisted  instruc- 
tion can  be  individualized  to  take  account  of  the  particular  strengths 
and  deficiencies  of  the  learner. 

These  assumptions  about  the  comparative  advantages  of  replacing  some 
portion  of  traditional  classroom  instruction  with  a  more  capital-intensive 
educational  technology  would  seem  especially  pertinent  to  the  case  of 
computer-assisted  instruction  (CAI) .    Recent  technological  breakthroughs  in 
computers  and  particularly  the  advent  of  mini-computers  and  inexpensive 
memory  devices  have  both  expanded  the  capability  and  flexibility  of  com- 
puters with  respect  to  their  instructional  applications  while  reducing 
their  costs  considerably.    Also,  CAI  permits  a  large  variety  of  methods 
for  individualizing  instruction  according  to  the  actual  performance 
of  the  learner.     For  example,  a  computer-based  curriculum  can  be  designed 
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to  provide  automatically  additional  problems  for  a  student  in  any  area 

in  which  he  is  not  performing  according  to  some  pre-set  standard,  or  it 

can~be  arranged  in  particular  sequences  of  instructional  tasks  that 

emphasize  his  special  instructional  needs* 

Despite  the  promise  of  educational  technology  in  improving  educational 

outcomes  and  reducing  costs,  there  is  little  supporting  evidence  of  a 
rigorous  nature  on  either  the  relative  costs  or  educational  results.^ 
In  response  to  this  evaluative  deficiency,  the  National  Institute  of 
Education  decided  to  undertake  an  experimental  study  of  computer-assisted 
instruction  in  order  to  evaluate  its  effects  on  the  improvement  of 
reading,  language  skills,  and  arithmetic  operations  of  elementary  school 
children.     The  experiment  was  initiated  in  the  Fall  of  1976  on  the  basis 
of  a  research  design  that  was  prepared  by  the  EducationaI>>sTesting  Service 
(ETS)  and  implemented  in  the  Los  Angeles  Unified  School  District  (LAUSD) . 
Known  as  the  ETS/LAUSD  Study  on  Computer-Assisted  Instruction  and  Com- 
pensatory Education,  the  study  was  intended  to  ascertain  both  the  effects 
of  a  particular  computer-basedL  instructional  system  and  curriculum  on 
student  test  scores  in  three  subject  areas  as  well  as  the  costs  for 
replicating  this  particular  system. 

With  respect  to  educational  effectiveness,   the  research  design  was 
constructed  in  order  to  ascertain  the  effects  on  test  scores  in  reading, 
arithmetic,  and  language  arts  of  the  "drill  and  practice"  curriculum  of 
the  Computer  Curriculum  Corporation  (CCQ)  among  students  at  different 
elementary  grade  levels.     The  evaluation  was  arranged  to  determine  the 
effects  of    10  minute  daily  sessions  of  CAI  on  student  achievement. 
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Comparisons  of  test  results  for  disadvantaged  students  are  being  made 
according  to  the  number  of  daily  sessions  of  CAI,  the  subjects  in  which 
CAI  sessions  were  given,  and  the  number  of  years  in  which  students 
received  CAI.    Hopefully,  the  studies  of  effectiveness  will  reveal  the 
educational  impact  of  this  particular  CAI  approach  across  subject,  grade 
levels,  amounts  of  exposure,  and  different  types  of  students  (race,  sex, 
ethnicity,  social  class  origin,  and  so  on).^ 

^Obviously,  the  evaluation  of  the  effectiveness  of  this  CAI  approach 
does  not  address  the  issue  of  costs.     Given  its  focus  on  the  educational 
needs  of  disadvantaged  students,  there  are  two  questions  pertaining  to 
costs  that  arise.,    The  first  question  is  bas,ed  upon  the  assumption  that 
funding  for  special  educational  services  for  disadvantaged  students  is 
derived  primarily  from  special  categorical  aid  for  that  purpose,  such 
as  that  received  under  Title  I  of  the  Elementary  and  Secondary  Education 
Act  of  1965.     Therefore,  it  is  imEOXtant  to  know  if  the  CAI  can  be  pro- 
vided within  the  budget  that  is  available  for  these  compensatory  educa- 
tional services  for  disadvantaged  youngsters.     Second,  it  is  important 
to  know  if  the  CAI  approach  can  improve  the  educational  proficiencies 
of  disadvantaged  students  at  costs  that  are  similar  or  less  than  those 
associated  with  other  instructional  alternatives. 

The  first  issue  is  one  of  cost-feasibility.     If  the  costs  of  this 
CAI  approach  exceed  the  funds  available  for  instructional  purposes  for 
disadvantaged  youngsters,  it  will  not  be  within  the  boundaries  of  feasi- 
bility.    The  second  issue_is  one  of  cost-effectiveness.     Even  if  the 


CAI  can  be  provided  within  the  pr^s^t  budgets  for  compensatory  educa- 
tion, it  should  be  adopted  only  if  it  provides  better  results  relative 
to  its  costs  than  existing  alternatives. 
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Cost-feasibility  can  be  examined  by  evaluating  the  costs  of  the  CAT. 
instruction  and  ascertaining  whether  it  is  within  the  budgetary  allocations 
provided  for  compensatory  education  by  Title  I  of  the  Elementary  Secondary 
Education  Act  of  1965  or  by  various  .state  and  local  compensatory  programs. 
Cost-effectiveness  comparisons     can  only  be  made  by  comparing  both  the 
results  of  the  CAI  approach  and  it/5  costs  with  the  results  and  costs  of 
other  instructional  alternatives.    While  this  study  .can  establish  its 
costs,  it  is  not  designed  to  pursue  its  effects.    However,  the  overall 
CAI  experiment  on  which  this  study  is  based  will  provide  rather  sophis- 
ticated estimates  of  test  score  results  associated  with  student  exposure 
to  different  amounts  of  CAI  and  different  subjects.    Accordingly,  the 
costs  that  are  estimated  in  this  study- can  be  combined  with  the  experi- 
mental effects  of  the  CAI  for  cost-effectiveness  comparisons  with  other 
instructional  approaches. 

In  this  paper  we  will  estimate  the  replication  costs  of  the  CAI 
approach  used  ^n  the  LAUSD/ETS  experiment,  that  is  the  cost  of  replicating 
that  system  'in  other  school* settings.     In  doing  this  we  will  limit  those 
costs  only  to  opes  that  are  associated  with  the  delivery  of  the  CAI  instruc- 
tion, while  omitting  costs  that*  are  tied  uniquely  to  the  experimental 
status  of  the  present  system.     That  is,  we  are  concerned  with  the  costs 
of  introducing  this  particular  CAI  approach  into  other  schools  outside  of 
the  present  experimental  situation.    At  the  same  time,  we  are  concerned 
w4^h' nrOcJifications  of  the  experimental  CAI  that  might  affect  costs.  In 
particular,  there  exists . a/later  version  of  the  present  computer  that 
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is  more  advanced.     The  cost  implications  of  the  nev;er  computer  will  be 
examined  after  exploring  the  costs  of  replicating  the  present  experi- 
mental approach. 

The  organization  of  the  paper  will  be  as  follows:     First,  a  brief 
description  will  be  given  of  the  present  CAI  system  and  its  configuration 
in  the  LAUSD/ETS  schools.     Second,  a  short  presentation  will  be  made  of 
the  costing  methodology  that  will  be  used  in  this  study.     Third,  cost 
estimates  for  replicating  the  present  CAI  system  will  be  made.  Fourth, 
the  cost  feasibility  of  adop,tipg  this  system  of  CAI  for  compensatory 
education  will  be  evaluated  as  well  as  the  cost  implications  of  a 
more  advanced  system. 


THE  SYSTEM  OF  COMPUTER-ASSISTED  INSTRUCTION 
The  purpose  of  this  section  is  to  provide  a  brief  summary  of  the 
implementation  of  computer-assisted  instruction 'in  the  ETS/LAUSD  study. 
This  description  is  of  special  importance,  because  each  CAI  approach  and 
installation  is  associated  with  different  resource  costs  and  effects. 
The  ETS/LAUSD  experiment  is  based  upon  the  ijse  of  a  particular  computer 
system  and  curriculum  that  have  been  utilized  in  a  specific  way.  There- 
fore, it  is  important  to  provide  some  description  of  the  system  and  its 
application.     It  is  equally  important  to  bear  in  mind  that  the  evaluation 
of  this  particular  CAI  approach  with  respect  to  costs  or  educational 
effects  can  not  necessarily  be  generalized  to  other  CAI  approaches. 
Rather,  all  of  the  results  will  be  limited  to  the  specific  CAI 
application  that  is  being  evaluated. 
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-  The  heart  of  the  ETS/LAUSD  instructional  approach  is  the  use  of  the  - 
A-16  computer  for  providing  drill  and  practice  instruction  for  the  students. 
Students  are  seated  at  terminals  which  consist  of  a  keyboard  (reasonably 
similar  to  that  of  a  typewriter)  and  a  cathode  ray  tube  (or  CRT  which  is 
similar  to  a  television  screen).     Each  A-16  can  be  used  to  service  "up  to 
32  terminals,  simultaneously.     The  A-16  contains  curricula cfor  all  of  the 
elementary  grades  for  each  of  the  three  subject  areas:  mathematics,  reading, 
and  language  arts.     Each  session  la^ts  for  ten  minutes , ^although  some  stu- 
dents may  be  assigned  to  undertake  more  than  one  session  per  day. 

Each  student  "signs-on"  at  his/her  terminal,  and  begins  the  session 
where  he  or  shehad  left-off  in  the  previous  session.     A  problem  is  displayed 
on  the  CRT,  typically  in  a  multiple-choice  or  in  a  "fill  in  the*^ blank"  for- 
mat.    For  example,   the  student  ;might  be  given  a  problem  in  arithmetic  opera- 
tions such  as  vertical  addition  or  subtraction,  and  he  or  she  must  type  in 
the  solution.     Or,  the  student  might  be  asked  to  fill  in  the  correct  form 
of  a  verb  in  a  sentence.     If  the  answer  is  correct,  anasterix   is  displayed 
on  the  CRT;  if  it  is  incorrect,  the  student  is  so-informed.     In  either  cas*e, 
a  new  problem  is  displayed.    When  a  student  aqhieves ^adequate  proficiencies 
on  a  particular  part  of  the  curriculum  —  as  evidenced  by  a  high  enough- 
proportion  of  correct  answers  —  the  system  provides  problems  at  a  higher 
level  of  difficulty  of  that  type.     The  curriculum  is  not  designed 
to  introduce  new  material  as  much  as  it  is  to  provide  an  opportunity  to 
practice  concepts  that  have  already  been  taught. 

There  are  two  principal  personnel  who  assist  the  students  in  working 
with  the  CAI  system.        A  coordinator  is  responsible  for  the  entire  operation 
in  a  particular   school  including  the  scheduling  of  students;  the  provision 
of  summaries  of  progress  for  each  class  to  the  classroom. 


teacher^  (available  from  a  printer  that  is  attached  to  theA-16);  the 
security  and  condition  of  the  equipment  (such  as  insuring  that  the  equip- 
ment is  working  properly. and  calling  maintenance  personnel  when  necessary); 
and  the  overall  supervision  of  the  students  in  working  at  the  terminals. 
The  coordinator  is  assisted  by  a  teaching  aide  who  monitors  the  students 
and  answers  their  questions  or  Assists  them  when  they  seem  to  be  having 
difficulties.  . 

'  The  LAUSD/ETS  experiment  was  based  upon  using  four  experimental 
schools  and  two  comparison  ones  that  would  not  receive  the  CAI.  Two 
of  the  four  experimental  schools  were  large  enough  that  they  could 
>  utilize  anA-16  with  a  full  complement  of  32  terminals.     The  other  two 
schools  had  smaller  student  populations,  so  they  shared  anA-16  through 
the  use  of  telephone  lines  and  special-  equipment  (multiplexer  and  modems). 
Each  of  these  schools  had  16  terminals  installed,  so  that  the  shared 
A-16  was  also  attached  to  a  total  of  32  terminals.    The  CAI  rooms  had  to  be 
modified  to  accomodate  the  special  configuration  of  equipment  as  well  as 
to  assure  security  and  an  appropriate  climate  of  temperature  and  humidity 
for  maintaining  the  computer. 

COSTING  METHODOLOGY 
The^concept  of  costs  typically  tends  to  be  confusing  to  evaluators. 
Often,  the  tendency  is  to  review  budgets  to  estimate  the  costs  of  a 
particular  project.    But  the  costs  that  one  finds  in  a  budget  or  accounting 
statement  are  often  in  error  or  are' misleading  for  a  number  of  reasons: 
First,  budgets  typically  show  estimated  costs  rathe^  than  actual  ones. 
To  the  degree  that  there  are  discrepancies  between  the  real  costs  and 
the  estimated  ones,  budgetary  costs  will  not  be  accurate.     Second,  budgetary 
costs  often  provide  costs  of  resources  that  will  be  used  over  different 
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time  periods.     For  example,  while  salaries  in  a  given  year  will  cover  the 

labor  services  during  that  period,  a  piece  of  equipment  may  be  utilized 

for  many  years.    Yet,  the  cost  will  be  assigned  only  to  the  year  in 

which  the  equipment  was  purchased,  when  it  should  be  divided  over  the 

entire  period  of  use  on  an  annualized  basis.     Third,  costs  of  contributed 

inputs  are  not  included  in  budgets,  confusing  the  question  of  what  are 

the  true  costs  of  a  project  with  the  question  of  who  paid  the  costs. 

Finally,  some  budgetary  costs  are  dis tor t'6'd  because  they  represent  special 

purchases  or  transactions  which  do  not  reflect  the  true  market  values 

of  the  transactions. 

A  more  appropriate  method  for  estimating  costs  is  to  use  the  ingre- 
3 

dients  method.      This  method  is  based  upon  the  assumption  that  whenever 
resources  that  have  alternative  uses  are  allocated  to  a  particular 
activity,  those  resources  have  a  cost  to  society.     The  cost  is  equivalent 
to  the  value  of  the  resources  in  their  most  productive  application. 
The  most  typical  way  of  estimating  these  costs  is  to  use  the  market  value 
of  the  resource.     Further,  in  order  to  obtain  annual  or  costs 
of  an  alternative,  the  costs  of  various  ingredients  that  are  utilized 
over  more  than  one  year  are  "annualized"  iti  order  to  charge  to  each  year 
only  the  costs  for  that  period  (rather  than  assigning  the  entire  cost 
tx)  the  year  of  purchase).     Since    there  are  sources  that  can  be  used -to 
evaluate  the  techniques  of  cost  analysis  within  this  framework,  we  will 
not  discuss  them  in  detail  here. 

The  following  steps  are  necessary  for  estimating  costs,  using  the 
ingredients  approach. 


1.  List. all  of  the  ingredients  or  resources  that  are 
required  for  implementating  the  instruction* 

2.  Estimate  the  costs  of  each  ingredient  on  the  basis  of 
actual 'costs  or  estimated  market  values* 

3.  Convert  costs  into  the  appropriate  categories  for  analysis 
such  as  annualized  costs,  average  costs,  marginal  costs, 
and  so  on* 

In  this  particular  case  we  wish  to  estimate  the  costs  for  replicating 
the  ETS/LAUSD  system  of  CAI  in  other  educational  settings,  and  we  wish  to 
evaluate  costs  under  different  organizational  arrangements* 
CAI  Ingredients  and  Their  Costs 

Before  enumerating  the  various  ingredients  of  the  CAI  system  and  their 
costs,  it  is  useful  to  mention  the  bases  on  which  ingredients  might  be 
classified  as  well  as 'the  sources  of  the  cost  information*     The  classifica^ 
tion  of  ingredients  can  be  done  in  any  way  that  is  functional  to  the  questions 
that  will  be  raised*    For  example,  one  could  classify  ingredients  under 
personnel,  facilities,  equipment,  and  miscellaneous  categories*    Or  one 
could  set  out  categories  of  ingredients  that  represent  fixed  investments 
as  well  as  those  that  represent  recurrent  cost  items*    The  main  criteria 
are  that  all  ingredients  are  accounted  for  in  the  classification  approach 
and  that  the  ultimate  categories  are  useful  for  analytical  purposes* 

The  derivation  of  cost  information  for  the  various  ingredients  will  be 
done  in  a  number  of  ways*    Where  budgetary  and  accounting  information  are 
appropriate,  they  will  be  used*    Where  such  cost  data  are  inappropriate 
or  misleading,  other  methods  of  obtaining  costs  will  be  utilized*     In  all 
cases,  the  sources  of  the  cost  information  will  be  specified  as  well  as 

the  methods  of  cost  estimation.     In  this  way,  the  reader  c^n  ascertain  how 
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the  costs  were  derived,  and  it  is  also  possible  to  modify  the  assumptions 
on  cost  estimation  to  determine  the  sensitivity  of  costs  to  different  premise 

COST  ESTIMATES 

For  purposes  of  cost  estimation^  the  ingredients  of  the  CAI  approach 
will  be  divided  into  six  categories:  (1)  Facilities  and  Equipment;  (2) 
Training;   (3)  Personnel;   (4)  Curriculum  Rental;  (5)  Maintenance;  and  (6) 
Miscellaneous  Factors.     Each  of  these  will  be  evaluated,  in  turn,  and  they 
will  be  combined  in  analyzing  the  overall  costs  of  CAI. 
(1)  Facilities  and  Equipment 

Any  CAI  approach  has  the  obvious  requirement  of  the  equipment  needed 
as  well  as  the  facilities   needed   to  provide  CAI  instruction.     In  the  case 
of  the  Los  Angeles  experiment,  the  equipment  for  a  school  using  a  single 
A-16  computer,  32  . terminals ,  and  a  printer  is  estimated  at  about  $121,000. 
Tlie  separate  breakdowns  for  each  type  of  equipment  are  shown  in  Table  1. 
That  table  also  presents  the  estimates  of  facility  costs.     These  include 
the  CO  t  cf  construction  of  a  normal  instructional  classroom  as  well  as 
the  renovations  that  must  be  made  to  accomodate  the  CAI.^  Renovation  costs 
include  special  carpentry  work,  protective  devices,  electrical  work  unique 
to  the  CAI  installation,  and  air  conditioning.     The  facilities  costs  are 
estimated  to  be  about  $68,500,  and  the  total  value  of  the  equipment  and 
facilities  is  assessed  af^a-lmost  $190,000  per  school. 

However,  we  are  not  concerned  with  the  total  costs  of  these  ingredients 
.as  much  as  we  are  with  their  annualized  costs.     That  is,  a  classroom  is 
assumed  to  have  a  life  of  25  years,  so  that  only  about  1/25  of  the  cost 
should  be  allocated  to  a  particular  annual  period.^  The  renovations  are 
assumed  to  have  a  life  span  of  ten  years,  and  the  equipment  is  estimated 
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TABLE  1  —  Facilities  and  Equipment 


Facilities; 

Cost  of  Construction  of  a  CAI  Room^                        $  50,000 
2 

Renovation  Cost  18,500 


68,500 


Equipmen  t :  , 

One  A-16  Computer  System  $  68,120 

Installation  3,000 

32  Hazeltine  Modular  I  terminals  Q  $1440/£^  46,080 

Delivery  @  $63/ea  2,016 

0 

One  Hazeltine  Thermal  Printer  1,950 

Delivery  23 


$  121,189 
TOTAL  $  189,689 


1.  It  was  reported  from  the  Educational  Housing  Branch  in  Los  Angeles 
that  to  replace  a  room  in  which  the  CAI  experiments  are  now  housed 
in  the  present  construction  market  will  cost  approximately  $50,000 
per  room. 

1 

2.  The  renovation  costs  include  counters ,  intrusion  alarm,  carpentry, 
paint,  electrical,  window  grilling,  air-conditioning  and  the  labor 

.  involved. 

3.  These  costs  are  derived  directly  from  the  CCC  contract. 
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to  have  a  life  span  of  six  years.     In  each  case  we  must  use.^a  standard 

approach  to  convert  the  overall  costs  into  annualized  ones,  where  the 

annualized  cost  represents  the  depreciation  and  interest  costs  foregone 

on  Che  investment  for  each  year.     Clearly,  the  annualized  cost  will  depend 

on  three  factors:  (a)  the  overall  investment  cost;  (b)  the  life  of  the 

facilities  or  equipment  or  the  amortization  period;  and  (c)  the  rate  of 

8 

Interest  on  the  investment  that  is  foregone. 

Table  2  shows  the  annualized  values  of 'facilities  and  equipment  costs 
with  the  specific  assumptions  about  the  amortization  period  and  three 
different  interest  rates.     Given  a  rate  of  interest  on  U.S.   treasury  bonds 
of  about  10  percent  at  the  present  time,  this  seems  to  be  a!  reason- 
able   figure  for  calculating  foregone  interest  on  the  invejstment.    On  ^ 
that  basis,  the  annualized  cost  of  facilities  is  about  $85^4  and  that  of 
equipment  is  about  $27,873.     Thus,  the  estimated  cost  of  facilities  and  • 
equipment  is  about  $36,397  per  year. 

(2)  Training  Costs  ■ 

I 

Training  costs  are  composed  of  two  types,  the  direct  costs  of  training 

■  •  [ 
-      ■  I  ■ 

and  the  indirect  costs.    The  direct  costs  are  the  most  obvious  ones,  con- 
sisting of  the  salaries  of  instructors,  costs  of  materials,  and  so  on.  The 
indirect  costs  refer  to  the  value  of  the  time  of  the  trainees.     In  the  case 
of  the  LAUSD  project,   the  direct  costs  of  training  were  included  in  the  costs 
of  equipment  by  CCC.     However,  the  indirect  ones  had  to  be  borne  separately. 
According  to  the  experience  of  CCC  personnel,  it  is  usually  sufficient  to 
provide  workshops  of  a  day  and  one-half  for  coordinators  and  half  a  day  for 
teachers.     The  cost  for  each  teacher  and  coordinator  will  vary  according 
to  experience .and  training  ahd  the  salary  levels  in  the  particular  school 
district.    However,  in  Los  Angeles  it  appears  that  salaries  and  fringe 
benefits  average  about  $20,000  for  a  school  year  that  is  not  more  ^than  200 
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TABLE  2  —  Annualizatlon  of  Facilities  and  Equipment  Costs 


Cost  Categories; 
Facility 

Construction  of  a  CAI  room 

Renovation 

Facility  Subtotal 


Amortization 
Period  (yrs) 


25 
10 


Cost 

$  50,000 
18,500 


07. 


Annualized  Cost 
107. 


157. 


$  2,000    $  5,508      $  7,7,50 
1,850        3,016  3,68^2 


$  68,500      $  3,850    $  8,524  $11,432 


Equipment 

Equipment  Subtotal 

TOTAL 


$121,189      $20,198    $27,873  $31,994 


$24,048    $36,397  $43,426 


1.   Refer  to  Table  1  for  the  details.     The  amortization  periods  for  all  computer 
related  equipment  are  assumed -to  be  6  years. 


TABLE  3  —  Annual  Personnel  Costs 


AdmiTvistration 

CAI  Coorditia^t^ 

Fringe  Benefits  orTabpve 
(3  16.7% 

Two  teaching  aides 

Substitutes 


$  1,965 
22,500 

4,086 
5,220 
780 


TOfAL 


$  34,551 
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days.  This  suggests  that  a  pay  rate  of  about  $100  a  day  is  an  appropriate 
basis  for  calculating  costs  of  the  time  required  by  teachers  and  coordina- 
tors  to  obtain  training.  Given  about  40  teachers  to  an  elementary  school, 
the  indirect  costs  of  teacher  training  are  about  $2000  for  a  half-day  work- 
shop and  about  $150  for  a  li5  day  workshop  for  the  coordinator.  Thus,  the 
ta|:al  estimated  indirect  costs  of  training  are  about  $2150. 

One  question  that  arises  is  hov^  this  figure  translates  into  an  annualized 
cost.\    It  is  unlikely  that  training  costs  of  this  magnitude  would  be  required 
for  each  year,  since  the  carryover  <&f  trained  teachers  ^nd  coordinators 
from  year-to-year  would  be  rather  high.    Yet,  any  turnover  of  teachers  will 
requ^ie  some  training  to  take  place  each  year,  even  if  it  is  merely  the 
coordinator  taking  the  teacher  away  from  his  or  her  classroom  duties  for 
half  a  day^for  instruction.    For  example,  with  a  turnover  xrate  of  ten  per- 
cent a  year,  about  four  new  teachers  would  have  to  be  trained  each  year 
a^t  a  co^t  of  about  $200.     In  fact,  after  the  first  year  this  would  be  the 

.only^cost  of  training  as  well  as, the  interest  foregone  on  investments  for 

\  \      ■  \ 

trainihg  in  previous  years.     If  we  use  those  two  components  to  estimate 

'^■■•.\  '       "  ■ 

costs,  the  total  indirect  training  costs  would  be  ten  percent  of  the  pre- 
vious investment  in  training  per  year  plus  the^costs  of  training  new  teachers. 
On  the  average,  ten  percent  of  the  training  investment  over  a  six  year 
period  would  be  about  $250  and  the  indirect  cost  of  training  four  new 
teachers  a  year  would  be  about  $200  for  a  total  of  $450  a  year.  Whatever 
the  assumptions  are  about  the  cost^  of  this  component,  the  overall  cost 
implications  are  so  small  that  they  will  have  little  impact  on  the  total 
cost  calculation^.^ 


(3)  Personnel  i  " 

Personnel  ingredients  for  the  CAI  demonstration  include  administrative 
resources,  the  CAI  coordinator,  two  teaching  aides,  and  substitutes  to 
cover  the  absences  of  the  coordinator.     These  are  shown  in  Table  3.  The 
function  of  the  administrative  personnel  is  to  negotiate  the  contracts  with 
the  companies  that  maintain  the  equipment,  to  arrange  payments,  and  to  pro- 
vide general  financial  and  logistical  administration  of  the  project.  The.^ 
annual  personnel  costs  for  this  function  were  estimated  at  $1965  on  the 
basis  of  previous  experience  of  the  Los  Angeles  schools  with  these  types  of  proj 

The  CAI  coordinator  is  responsible  for  the  overall  functioning  of  the 
CAI  instruction  including  the  scheduling  and  coordination  of  the  instruction, 
reports  to  teachers  on  student  progress,  and  the  monitoring  of  the  functioning 
of  the  equipment  and  i/ts  maintenance.     Especially  important  is  the  latter 
function,  since  equipment  failures  will  result  in  the  loss  of  instructional 
sessions.    Accordingly,  the  coordinator  must  be  aware  of  problems  and  the 
methods  of  getting  them  alleviated  by  the  appropriate  maintenance  personnel. 
Further,  the  cooridinator  must  work  closely  with  classroom  teachers  to  inte- 
grate the  drill  and  practice  sessions  of  CAI  with  classroom  work. 

In  the  LAUSD  case,  the  coordinators  were  so  carefully  chosen  and  so 
well-trained  that  they  needed  little  administrative  supervision  from  the 
school  principal  or  other  school  administrative  personnel.    Whether  this 
high  level  of  initiative  and  independence  could  be  maintained  in  a  replica- 
tion is  problematic.    However,  based  upon  the  success  of  coordinator  autonomy 
in  LAUSD,  we  have  not  indicated  any  supervision  in  the  cost  estimates.  The 
cost  of  the  coordinators  can  be  detertnined  directly  by  calculating  salaries 
and  fringe  benefits.     The  salary  component  was  estimated  at  $22,500  and  the 
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fringe  benefits  for  .that  portion  of  the  administrative  costs  and  the 
coordinator  were  $4086.     Fringe  benefits  do  not  apply  to  the  other  personnel 
categories  because  of  their  part-time  nature. 

Teaching  aides  monitor  the  performance  of  students  and  assist  them  in 
understanding  the  CAI  problems  and  in  solving  them.     Essentially,  they 
wander  among  the  students,  looking  for  situations  in  which  assistance  or 
supervision  is  needed.     Their  rate  of  pay  in  1977-78  was  $4.35  an  hour, 
and  it  takes  two  teaching  aides  working  about  600  hours  each  school  year 
to  assist  in  a  CAI  room  with  32  terminals..     This  particular  arrangement 
has  been  considered  highly  satisfactory  by  the  Los  Angeles  coordinators. 
The  total  cost  per  CAI  room  of  the  two  aides  is  about  $5220  a  year. 

The  final  personnel  cost  is  related  to  the  need  for  substitute 
teachers  to  undertake  the  coordination  functions  if. the  regular  coordinator 
is  ill.     Under  the  Los  Angeles  arrangements,  a  teacher  or  coordinator  can 
receive  up  to  12  days  a  year  in  paid  sick  leave.     Therefore,  provision  for 
up  to  12  days  of  substitute  teaching  at  about  $65  a  day  would  cost  about 
$780  per  year.    Based  on  these  amounts,  the  personnel  costs  per  year 
(for  1977-78)  totaled  about  $34,551. 
(4)  Curriculum  Rental 

The  curricula  that  are  used  for  the  CAI  approach  are  rented  from  CCC, 
the  company  that  provided  the  A- J.6  system.     The  rental  covers  the  cost  of 
using  the  three  sets  of  subject  curricula  in  mathematics,  reading,  and 
language  arts.    The  cost  of  the  rental  is  set  at  $204  a  year  for  each  of  the 
32  terminals  in  a  CAI  room  for  an  annual  total  of  $6528. 
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(5)  Maintenance  ^ 
The  provision  for  maintenance  of  the  equipment  is  arranged  through 

contrfih2<s  with  firms  that  specialize  in  such  care.     While  some  of  the 
maintenance  is  routine  and  periodic,  a  major  requirement  is  sers^ices 
of  an  emergency  nature  to  repair  malfunctions.     The  annual  cost  of  main-  ' 
taining  the  A-16  computer  is  $6120  a  ye^^r;  each  of  the  32  terminals  has 
a  maintenance  cost  of  $300  a  year,  or  $9600  for  all  terminals  in  a  CAI 
room;  and  the  thermal  printer  has  a  maintenance  cost  of  $360  a  year. 
The  total  cost  of  maintenance  is  about  $16,080  a. year. 

(6)  Miscellaneous  Factors 

Miscellaneous  cost  factors  include  insurance,  supplies,  and  the  costs 
of  energy  and  routine  maintenance  of  the  classroom.     The  appropriate  in- 
surance costs  are  those  that  are  incurred  by  virtue  of  the  existence  of 
the  CAI  approach.     These  would  include  the  additional  insurance  costs  for 
theft,  fire,  and  liability  attributible  to  the  CAI  facility  and  equipment. 
Of  these  components,  it  appears  that  liability  insurance  is  largely  unaffected, 
and  the  'impact  on  firfe  insurance  costs  is  not  readily  ascertainable.  How- 
ever, the  additional  theft  insurance  for  the  equipment  was  estimated  by 
the  Los  Angeles  school  authorities  at  abo.ut  $3,000  a  year  for  the  computer, 
32  terminals,  and  the  printer.     The  use  of  only  the  theft  component  may 
understate  slightly  the  true  insurance  costs,  by  omitting  the  fire  com- 
ponent.    However,  the  overall  omission  is  likely  to  have  a  relatively  small 
effect  on  total  costs,  since  insurance  represents  a  very  small  relative 
cost  item. 

Supplies,  enengy,  and  routine  maintenance  of  the  classroom  contain  many 
Items.   Supplies  include  the  typical  pencils,  paper,  books,  paper  for  the 
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printer,  and  so  on.     Energy  arid  telephone  costs  and  facility  maintienace  refer 
to  the  telephone  in  eaL\\  classroom  that  is  necessary  for  rapid  access  to 
maintenance  personnel  and  CCC-pin  case  of  breakdowns;  normal  heating, 
lighting,  and  pow^r  for  the  equipment;  ^nd  routine  cleaning  and  maintenance 
of  the  classroom.  \  Taken  together,  these  are  estimated  at  about  $3,000  per 


year.    Again,  even ^^ubstantial  changes  in  this  amount  (for  example  50%) 
would  liave  little  effect  on  overall  costs  per  student  session  because  of  the 
^relatively  small  magni-tijae  of  costs  for  the  category.     (Each  classroom 

is.  capable  of  providing  a  daily  session  on  an  annual  basis  for  over  700 

1  '        .       •  \  ■  ■ 

students  so  an  error  of  $]pOO  is  only  about  $2.00  per  session.) 
Siimmary  of  Annualized  Costs 

The  annualized  costs  in  1977-78  for  a  32  terminal  classroom  utilizing 
the  CCC  A*16  system  can  be  ^ummarized  in  the  following  tabulation. 

Facilities  and  Equipment  $  36,397 

/  ^  '  • 

^      Personnel  /  \  34,551 

/  i      Training  /  /  ■  450 

^            Curriculum  Ren tafl  6,528 

\  / 

^Maintenance       /  16,080 

Miscellaneous   I  6^000 


1  .  ■  \ 

Total  $100,006 


Rounding  off  this  estimate,  it  appears  that  it  costs  about  $100,000  a  year 
to  provide  a  classroom^  personnel,  equipment,  and  so  on  for  servicing  32 
terminals  with  this  pairticular  approach  to  CAI. 


Sl8 
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Average  Cost  Per  Session 

Given  thi3  total,  it  is  important  to  know  the  cost  per  session  on 
an  annual  basis  for  each  student,    ^hat  is,  what  is  the  cost  for  providing 
one  daily  session  of  ten  minutes  of  drill  and  practice  for  a  full  school 
year  to  each  student?    The  reason  that  this  particular  cost  figure 
is  important  is  that  it  would  enable  us  to  ascertain' the  cost-feasibility 
of  this  approach  to  CAI  as  a  method  of  providing  compensatory  education  to 
disadvantaged  youngsters,  by  comparing  the  amount  per  session  with  the 
average  amount  of  compensatory  funds  provided  by  the  federal  government 

under  Title  I  of  the  Elementary  and  Secondary  Education  Act  of  1965. 

i 

Clearly,  the  cost  per  session  depends  on  the  number  of  daily  sessions 
that  can  be  provided  by  the  CAI  system  on  an  annual  basis.     This  depends 
not  only  on  the  length  of  the  session,  but  also  on  the  organizational 
capacity  and  time  required  to  process  each  group  of  student  users.     That  is, 
there  niust  be  time  between  the  end  of  cne  ten  minute  session  and  the 
beginning  of  the  next  ror  one  group  of  students  to  sign  off  the  system  and 
return  to  class,  while  a  new  group  arrives,  is  seated,  and  signs  in. 
Finally,  the  number  of  sessions  will  also  depend  upon  the  overall  reliability 
of  the  equipment  and  its  operability  during  school  hours. 

In  theory,  the  system  could  be  used  for  up  to  six  and  one  half  hours 
a  day  during  regular  school  hours,  if  sessions  began  at  8:30  and  proceeded 
to  3:00  P.M.  with  no^ interruptions  ^for  lunch.     In  fact,   this  would  be  diffi- 
cult  to  dV organizationally,  since  time  is  needed  at  the  beginning  and/or 
end  of  the  day  to  accomplish  record-keeping  and  other  instructional  tasks 
assbciated  with  CAI.     Further,  it  would  be  difficult  to  coordinate  classes 
around  the  lunch  period,  and  a  "relief"  coordinator  would  be  needed  during 
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that  period.    With  respect  to  the  number  of  sessions  per  hout,  even  five 
sessions  of  ten  minutes  each  provide    only  about  two  minute  transition 
periods.'   Accordingly,  there  are  clear  limits  on  the  numbers  of  sessions 
that  can  be- accomodated.     Based  upon  the  actual  records  for  the  LAUSD 
system,  it  appeared  that  the  range  varied  from  21  sessions  to  25  sessions 
per  day,  with  a  median  of  about  23  sessions.     On  the  basis  of  these  experi- 
ences, we  can  estimate  the  cost  per  daily  session  per  student  for  a  school 
year. 


Number  of  Sessions  Annual  Cost  Per 
 Per  Day  Daily  Session 

Per  Terminal    For  32  Terminals 


21  .672  $  148.80 

23  736  135.90 

'  25  800  125.00 

Depending  on  the  number  of  sessions  per  day  for  each  terminal,  a  con- 
figuration using  the  A-16  and  32  terminals  in  a  single  classroom  can  accomo- 
date from  672  to  800  sessions  a  day.     Assuming  that  the  most  probable 
estimate  is  the  median  of  23  sessions  a  day  per  terminal,  736  sessions  can  be 
provided.     By  dividing  the.  numbet  of  sessions  by  the  $100,000  estimated 
annual  total  cost  for  this  CAI  configuration,  it  appears  that  the  annual 
costs  for  a  daily  session  of  ten  minutes  could  vary  from  about  $125  to 
almost  $150  per  year  for  one  daily  session  of  CAI.    The  estimate  for  23 

sessions  a  day  at  $136  is  probably  the  most  reasonable  one. 

Cost  Estimate'  for  the  Shared  System  '  ^ 

Before  comparing  that  cost  with  the  level  of  funding  available  for 
compensatory  education,  it  is  important  to  estimate  the  annual  cost  per 
daily  session  when  two  schools  share  an  A-16  system.     Clearly,  this  situa- 
tion presents  itself  when  there  is  not  an  adequate  student  enrollment  base 
in  a  particular  school  to  accomodate  some  700  or  so  daily  sessions.  It 
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could  also  be  evident  in  situations  where  only  a  particular  grade  level  or 
levels  utilized  the  CAI.    Of  course,  by  providing  multiple  daily  sessions,  (e.g. 
two  sessions  a  day),  an  A-16  could  be  utilized  to  full  capacity,  by  even  350  - 
400  students.     However  in  the  Los  Angeles  situation,  the  design  of  the  CAI 
experiment  meant  that  in  two  participating  schools  there  were  not  adequate 
students  assigned  to  CAI  to  fully  utilize  a  32  terminal  system  in  each  school. 
This  situation  provides  us  with  the  opportunity  to  ascertain  the  costs  of  a 
shared  CAI  computer. 

The  basic  configuration  for  the  shared  system  was  that  the  A-16  com- 
puter and  16  terminals  were  placed  in  one  school,  and  the  other  16  terminals 
were  placed  in  a  "sister**  school.     The  terminals  were  connected  to  the  first 
school  through  a  leased  telephone  line,  and  additional  equipment  was  required 
in  order  to  operate  the  sharing  arrangement. Table  4  shows  the  additional  costs 
incurred  for  a  shared  A-16  system.     With  the  shared  arrangement ,  two  class- 
rooms must  be  utilized  for  the  terminals  rather  than  one  classroom.  Based 
upon  the  costs  for  a  classroom  and  required  renovations  that  were  presented 
in  Table  2  and  replicated  in  Table  4,  the  total  cost  of  additional  facilities 
for  the  shared  arrangement  would  be  $68,500  which  would  be  about  $8,524  on 
an  annualized  basis.. 

The  additional  equipment  (two  modems  and  two  multiplexers)  and  their 
installation  have  a  cost  of  almost  $12,500  which  translates  into  an  annualized 
cost  of  about  $2,866.     Taken  together  the  additional  outlay  for  the  shared 
facilities  and  equipment  is  almost  $81,000  which  translates  into  an  annualized 
cost  (using  a  10  percent  interest  rate  on  the  (undepreciated  portion)  of 
$11,390.    With  respect  to  personnel  for  the  shared  arrangement,  we  assume 
that  the  administrative  costs  for  making  financial  arrangements  and  monitoring 
contracts  is  roughly  equivalent  to  the  single  school  approach.    However,  an 
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TABLE  4  —  Additional  Costs  Incurred  for  Shared  A-16  Syst 


em 


Cost  Categories; 

Facili ty 

Construction  of  a  CAI  room 
Renovation 


SUBTOTAL 


Amortization 

Period  (yrs)  Cost 


25 
10 


$  50,000 
18,500 

$  68,500 


Annualized  Cost 
10% 


$  5,508 
3,016 


$  8,524 


Equipment 

TWO  Modems 

Two  Multiplexers 

Installation 


6 
6 
6 


$  4,710 
7,550 
200 


12,460 


2,866 


TOTAL 


$  80,960 


$  11,390 


Personnel 

One  Coordinator 

Fringe  Benefits  on  above  (316.7% 
Substitutes 

SUBTOTAL 


22,500 
3,758 
390 

$  26,648 


Maintenance 
Printer 


Miscellaneous 


GRAND  TOTAL 


$  80,960 


$  360 
$  3,000 

$  41,398 


additional  coordinator  is  needed  for  the  classroom  in  the  shared  configura- 
tion, and  additional  provision  for  substitutes  is  necessary.     These  are 
estimated  to  cost  about  $26,648  per  year.     The  training  cost  for  the  addi- 
tional coordinator  is  so  small  that  it  is  inconsequential  (about  $150 'for 
the  day  and  one  half  of  salary)  and  will  not  be  included  in  the  total; 

Additional  costs  of  maintenance  seem  to  affect  only  the  additional 
printer  at  $360  a  year  and  the  modems  and  multiplexers  are  maintained  on 
the  basic  CCC  contract,  so  their  costs  can  not  be  easily  broken  out. 
Miscellaneous  costs  include  the  telephone  line  between  schools,  routine 
maintenance  of  the  facilities,  lighting,  heating,  electric  power,  and 
supplies.-  These  are  estimated  at  about  $3000  per  year,  and  insurance  costs 
are  not  affected  by  distributing  the  terminals  between  the  two  schools. 

When  these  additional  costs  of  the  shared  arrangements  are  totaled, 
about  $41,400  is  added  to  the  total  cost  in  comparison  tJith  the  single  class- 
room, 32  terminal,  A-16  approach.    Again,  assuming  23  daily  sessions  per 
terminal  and  a  total  cost  of  about  $141,000  per  year  for  the  shared  system, 
the  annual  cost  per  daily  session  of  CAI  instruction  is  about  $192.  In 
other  words,  the  shared  system  increases  the  cost  per  session  by  about  40 
percent  or  $56. 

COST  FEASIBILITY 

Are  these  costs  high  or  low.?-' Clearly  that  depends  on  what  the  costs  are 
buying  in  terms  of  educational  services  and  effectiveness  in  relation  to  what 
spending  those  funds  on  alternatives  might  produce.     Such  cost  effectiveness 
comparisons  are  absolutely  essential  in  using  cost  information  to  ascertain 
whether  a  particular  educational  technology  or  other  instructional  approach 
is  a  good  investment.     However,  we  lack  both  the  cost  of  other  alternatives 
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and  effectiveness  data  on  CAI  versus  other  alternative's  for  this  paper.  Some 
of  those  data  will  be  forthcoming  at  the  completion  of  the  CAI  experiment  and 
can  be  drawn  upon  for  cost-effectiveness  comparisons  at  that  time. 

The  purpose  of  cost  feasibility  analysis  is  much  .more  modest.     It  simply 

0 

a§ks  if  the  costs  of  the  instructional  approach  can  be  accomodated  within 
the  limits  of  the  budget  assigned  for  such  purposes.     In  order  to  answer 
cnat  question,  we  will  wish  to  compare  the  costs  of  CAI  with  the  level  of 
funding  provided  for  compensatory  education  by  Tit 1^  Ijof^tU  Elementary  and 
Secondary  Education  Act  of  1965.     That  is,  presumably  the  CAI  system  that  is 

being  evaluated  is  addressed  primarily  to  drill  and  practice  for  remediation. 

i  '  ' 

Accordingly,  this  would  seem  to  be  the  imost  relevant  framework  for  a  cost  - 
feasibility  analysis.  I  ^  / 

In  fiscal  year  1977,  Title  I  had  appropriations  of  about  $2  billion  for 
about  5  million  youngsters.     This  meani^  that  on  theLverage  about  $400  was  pro- 
vided for  each  of  the  students  covered  by  the  program.     Cleatly,.  not  all  of 
this  was  allocated  to  classroom  instruction.    Some  was  expended  on  admini- 
stration, health  and  'diagnostic  services,  nutrition,  and  so  on.  However, 
we  will  assume  that  about  $400  per  pupil  repres^\ts  an  upper  limit  for  | 
compensatory  education  in  the  classroom.     Using  this  as  a  basis  for  cost  I 

feasibility,  $400  would  cover  about  three  daily  sessions  of  CAI  at  $136  | 

i 

per  session  with  32  terminals  to  a  classroom  or  two  sessions  at  $192  under  - 
the  shared  arrangement.     This  means  that  all  three  curricula  could  be  pro- 
vided under  the  lower  cost  configuration  or  two  could  be  provided  under 
the  higher  cost  one.     It  also  means  that  two  curricula,  for  example  reading 
and  mathematics,  could  be  provided  under  the  l|wer,  cost  option,  while  allowing 
the  remaining  $128  per  student  to  be  used  fordther  purposes.     On  this  basis, 
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one  would  conclude  that  the  CAI  appraoch  that  has  been  evaluated  meets  a 

general  cost-feasibility  test.    That  is,  iri  is  feasible  to^\consider  this 

approach  within  ^he  constraints  of  existing;  provisions  for  compensatory 

education.  i     \  I 

U '  1  .  '       *  ' 

COSTS  OF  A  MORE  ADVANCED  SYSTEM 

One  of  the  major  questions  that  arises  in  evaluating  the  costs  of  a 

changing  technology  Is  the  direction  and  magnitude  of  fu,ture  costs  based 

upon  more  advanced  approaches.     This  is  particularly  important  in  any 

strategy  based  upon  computers,  since  the ; technology  of  mini-computers  and 


memory 


devices  has  been  developing  at  a  rapid  pace  with  drastic  reductions 


in  th^  cost  of  any  given  capability.     Clearly,  the  Ijonger  run  situation 
would 
would 


suggest|  that  at  least  the  cost  of  equipment  wijth  a  given  performance 
decline,  and  it  is  important  to  ascertain  the  impact  of  these  potential 


equipmipnt  cost  declines  on  the  overall  costs  of  CAI  instruction. 

However,  before  examining  some  evidence  on  this  question,  it  is  important 
to  point  out  a  phenomenon  which  is  typically  overlooked  in  predicting  cost 
changes  of  technological  innovation.     The  annualized  costs  of  all  the  com- 
puter equipment  including  the  terminals  represented  only  about  28  percent 
of  total  annualized  costs, as  evidenced  by  comparing  the  costs  of  $27,873  in 
Table  2  with  the  total  costs'  of  $100,000  for  a  32  terminal  classroom.  This 
means  that  even  a^'ather  drastic  reduction  in  the  28  percent  of  the  cost 
accounted  for  l^y  equipment  will  amount  to  a  much  smaller  reduction  in  the 
total  cosjtl^ ;  For.^example,  if  the  cost  of  equipment  declined  by  one  third, 
total'costs  would  decline  by  less  than  ten  percent.    At  the  same  time,  the 
costs  of  personnel,  maintenance,  construction  and  other  personnel  intensive 
categories  are^  rising  rap  i(/ly,  at  least  offsetting  partially  the  potential 
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dec lines ^ in  the  cost  of  computer  hardware.    Accordingly,  it  is  important 
tj>  recognize  that  there  will  be  inherent  limits  to  cost  reduction  for  CAI, 
eyen  with  rapid  technological  improvements  in  hardware. 

In  the  particular  case  of  the  A-16  system,  we  were,  fortunate  in  that 
/CCC  had  developed  a  more  advanced  CAI  approach  during  the  impltementation 
phase  of  the  LAUSD  experiment.     The  more  advanced  computer  is  the  CCC-17 
which  can  drive  some  96  terminals  rather  than  the  32  terminals  to  which  the 
A-16  is  limited.     CCC  also  claims  that  the  17  is  more  flexible  and  productive 
than  the  A- 16  for  a  number  of  reasons.     First,  it  uses  special  terminals 
provided  by  CCC  which  permit  more  flexible  design  and  format  of.curricula 
as  well  as  a  wider  variety  of  interactive,  feedback  responses  between  the 
pupil  and  the  computer.     Second,  the  central  processing  Wit  has  greater 
capacity  for  storing  additional  curricula  and  can  process\curricula  of  a 
wider  variety  than  the  A-16.     For  these  reasons  the  CCC- 17  may  also  be  more 
effective  for  each  session  that  the  A-16,  although  that  is  ultimately  an 
empirical  issue  rather  than  a  theoretical  one.     CCC  has  provided  the  CCC-17 
for  one  classroom  for  the  final  year  of  the/ LAUSD/ETS  experiment,  so  some 
empirical  data  should  be  forthcoming  on  this  issue. 

However,  the  purpose  of  this  investigation  is  to  ascd,rtain  the  cost 
per  session  of  the  newer  technology.     Sincfe  the  CCC-17  repl^esents  a  lar- 
ger system  capable  of  supporting  96  terminals,  we  will  estii\^te  the  costs 
of  using  a  single  CCC-17  for  providing  CAI  to  three  classrooms  of  32  ter- 
minals.    This  will  enable  us  to  ultimately  compare  the  costs  of  the  CCC-17 
for  96  terminals  with  that  of  the  A-16  on  a  32  terminal  classroom  basis. 

Table  3  shows  the  estimated  total  and  annualized  costs  of  both  the 
facilities  and  equipment  for  the  CCC-I7  configuration.     The  cos^t  of  the 

I        -  ' 
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TABLE  5  —  Annualized  Cost  for  Three  (3)  Schools 
Sharing  the  CCC-17  System 


Cost  Categories! 


Facilities 


Amortization 
Period  (Yrs) 


Construction  of  CAI  Room  25 
Renovations  10 

SUBTOTAL 


Cost 


$  150,000 
55,000 


Annualized  Cost 
0%  10%  15% 


$    6,000    $  16,500  $  23,250 
5,550         9,047  11,045 


$  205,500       $  11,550    $  25,547  $  34,295 


Equipment  i 

Computer-Related  Equipment 
(includes  terminals) 

Installation 


.6 
6 


$  314,814 
13,800 


SUBTOTAL 


$  328,614        $  54,769     $  75,581  $  86,754 


TOTAL  FACILITIES  AND  EQUIPMENT  $534,114         $66,319     $101,128  $121,049 
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facilities  component  is  identical  to  that  shown  in  Table  2  except  that  it 
is  based  upon  three  classrooms  rather  than  one  classroom.     (Of  course  we 
will  evaluate  the  costs  per  session  based  upon  the  larger  number  of  ter- 
minals serviced  by  the  CCC-17  to  make  the  cost  estimates  comparable  on  a 
.student  session  basis.)    The  equipment  costs  include  the  CCC-17  system, 

0 

96  terminals,  a  cluster  controller  for  every  32  terminals  which  provides 

power  to  the  terminals  and  routes  information  between  the  computer  and 

terminals,  a  printer  for  each  school,  modems  for  remote  schools,  and 

tables  for  each  CAI  room.     All  of  the  cost  figures  are  taken  from  pub- 

lished  documents  furnished  by  the  marketing  office  of  CCC  (dated  April 

17,  1978).     Total  facilities  and  equipment  costs  are  $534,114  or  about 

$101,128  in  annualized  costs  when  the  interest  rate  on  the  undepreciated 

investment  is  ten  percent. 

Personnel  costs  and  the  indirect  costs  of  training  were  calculated 

« 

in  the  same  manner  for  the  CCC-17  configuration  as  for  the  A- 16,  except 
that  they  are  shown  for  three  classrooms.     These  and  other  costs  are" 
reflected  in  Table  6.     Curriculum  rental  was  estimated  by  CCC  at  $20,857 
and  maintenance  at  $42,072.     The  miscellaneous  costs  are  also  similar  to 
those  calculated  for  the  A-16.    The  total  of  all  of  these  components  is 
$181,931  and  when  the  annualized  costs  of  the  equipment  and  facilities 
of  $101,128  are  added,  the  total  annualized  cost    Of  the  CCC-17  is  esti- 
mated to  be  $287,059.     In  order  to  find  the  average  cost  per  session, 
we  need  only  divide  this  annual  cost  by  the  nomber  of  daily  session  pro- 
vided on  an  annual  basis.     This  is  shown  under  different  assumptions  about 
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TABLE  6  —  Annual  Costs  of  Personnel,  Training,  Curriculum  Rental, 
Maintenance  and  Miscellaneous  Components  of  CCC-17 


Personnel 

Administration 
Coordinators 

Fringe  Benefits  on  above  @16.7% 
T.A.'s 

Substitutes 

SUBTOTAL 

Training  (indirect  costs) 
Curriculum  Rental 


$  5,895 
67,500 
12,257 
15,660 
2,340 

$  103,652 

$  1,350 
20,857 


Maintenance 


42,072 


Miticellaneous 


TOTAL 


18,000 
$  185,931 
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the  number  of  daily  sessions  provided: 

Number  of  Sessions  Annual  Cost  Per 

Per  Day  Da.-<  Ty  Session 

Per  Terminal       Per  96  Terminals 

21  2016  $  142.30 

23  2208  130.00  • 

25  2400  119.60 

Based  upon  the  median  number  of  23  daily  sessions,  the  average  cost  per 
session  for  the  CCC-17  is  estimated  to  be  about  $130  in  comparison- with 
about  $136  for  the  A-16.^ 

This  suggests  that  the  CCC-17  has  a  cost  that  is  about  fiv^  percent 
lower  per  CAI  session  than  the  A- 16.     This  represents  a  rather  small 
difference,  especially  since  it  assumes  that  the  CCC-17  is  utilized  to 
capacity.     One  of  the  advantages  of  the  smaller  scale  of  the  A-16  is  that 
it  provides  somewhat  more  flexibility.     Since.it  can  be  utilized  in  multi- 
ples of  32  terminals,  there  is  likely  to  be  less  of  a  problem  in  under- 
utilization  than  a  system  that  must  be  implemented  in  multiples  of  96 
terminals.    Because  of  the  high  fixed  costs  of  these  types  of  systems, 
underutilization  hardly  reduces  total  costs  at  all.  This  means  that  one  must 
divide  relatively  irreducible  total  costs  over  fewer  sessions,  with  a  marked 
rise  in  cost  per  session.     For  that  reason,  the  five  percent  reduction 
in  cost  per  session  under  assumptions  of  full  utilization  would  deteriorate 
rather  quickly  if  the  CCC-17  could  not  be  fully  utilized  at  a  scale  of 
96  terminals. 

One  o.ther  point  that  ought  to  be  emphasized  is  that  of  the  total 
annual  cost  of  $287^000  for  the  CCC-17,  only  about  $76,000  is  accounted 
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for  by  t/he  cost  of  the  computer  hardware.     This  means  that  almost  three 
quarters  of  the  cost  is  allocable  to  factors  that  are  not  ostensibly 

a 

affected  by  improvements  in  computer  technology,  thus  limiting  the  cost 
savings  obtainable  by  technological  advances  in  the  CAI  system.     In  fact, 
as  ^  general  rule,  virtually  all  technologically-based  instructional 
systems  will  show  only  about  one  quarter  to  one  third  of  the  costs  are  associ- 
ated with  their  "hardware."    This  means  that  drastic  reductions  in  the 
costs  of  such  hardware  may  have  only  nominal  effects  on  overall  costs  of 
the  instructional  strategy.     Further,  to  the  degree  that  the  decrease  in 
even  those  costs  is  associated  with  a  larger  scale  of  operation,  even 
these  cost  reductions  may  not  be  realized  unless  the  system  can  be  utilized 
to  full  capacity. 

It  should  be  noted  that  according  to  CCC,  the  CCC-17  is  educationally 
superior  to  the  A-16.     Clearly,  the  cost  per  session  is  not  as  important 
as  the  cost  per  unit  of  educational  effectiveness.    Thus,  even  if  the  costs 
of  the  CCC-17  are  comparable  to  those  of  the  A-16,  a  superior  level  of 
effectiveness  may  still  make  it  a    better  investment.    However,  without 
data  on  the  relative  effectiveness  of  the  two  systems,  it  is  impossible 
to  evaluate  this  claim. 

SUMMARY 

The  purpose  of  this  paper  was  to  estimate  both  the  costs  and  cost  - 
feasibility  of  utilizing  a  particular  CAI  approach  for  compensatory  educa- 
tional purposes.     The  particular  approach  that  was  chosen. is  the  CCC  A-16. 
and  its  impleme^tation  for  a  four  year  experiment  on  the  effectiveness^  of 
CAI  that  had  been  established  in  the  Los  Angeles  Unified  School  District. 
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Based  upon  the  Ingredients  approach  to  cost-analysis,  it  was  found  that 
up  to  three  sessions  of  drill  and  practice  of  ten  minutes  duration  could 
be  provided  for  each  disadvantaged  child  at  the  present  level  of  Title  I 
expenditures.     This  means  that  three  different  subjects  could  be  provided 
or  that  multiple  sessions  in  one  or  two  subjects  could  be-  offered  for 
each  child.     As  such,  it  appears  that  the  instructional  strategy  is  cost  - 
feasible  within  present  provisions  for  compensatory  education.  Utilizing 
the  A- 16  between  schools  would  increase  costs  rather  substantially ,  but 
two  sessions  of  CAI  would  still  be  feasible  within  present  compensatory 
educational  allocations. 

Costs  were  also  estimated  for  the  more  advanced  CCC-17  computer 
system,  and  somewhat  surprisingly  the  costs  were  in  the  same  range  as 
those  of  the  A- 16.     In  part,  this  finding  reflects  the  very  heavy  "soft- 
ware" components  of  CAI  approaches,  and,  in  part,  it  may  reflect  the 
possibility  thar  the  CCC-17  is  more  effective  than  the  A- 16  (even  though 
the  costs  are  quite  similar).     It  is  clear  that  a  more  exhaustive  analysis 
of  the  merits  of  different  CAI  approaches  as  well  as  a  comparison  between 
them  and  other  instructional  strategies  will  require  effectiveness  data 
as  well  as  cost  estimates.     Some  of  these  should  be  forthcoming  from  the 
ETS/LAllSI)  experiment,  and  it  is  hoped  that  a  cost-effectiveness  comparison 
can  be  made  at  some  future  date. 
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FOOTNOTES 


1.  The  best  studies  in  this  area  are  Jamison  et  al .  1976  and  1970  with 
respect  to  CAI.    However,  cost-effectiveness  analyses  of  other  techno- 
logies can  be  found  in  Instructional  Science  1975.     See  Carnoy  and 
Levin  1975  for  a  critique  of  the.  methodologies  of  these  studies. 

2.  ^These  studies  will  be  forthcoming  in  1980  by  Paul  Holland,  Dean  Jamison, 

and  Majge  Ragosta  of   Educational  Testing  Service. 

3.  Virtually  all  of  the  issues  discussed  here  are  reviewed  methodologically 
in  Levin  1975.    The  best  application,  of  costing  methodologies  to  instruc- 
tional technologies  is  Jamison  et  al, ,  1978. 

4.  Ibid.    This  paper  will  ffot  include  student  time  as  a  resource,  since  it 
is  difficult  to  place  a  value  on  this  dimension.    However,  alternative 
instructional  strategies  with  mostly  different  demands  on  student  time 
should  take  this  component  into  account. 

5.  As  school  enrollments  decline,  it  is  conmion  for  some  observers  to  ques- 
tion whether  any  cost  should  be  attached  to  newly  available  classrooms 
that  are  no  longer  needed  to  service  regular  enrollments.    However,  such 
facilities    are  not  costless  as  long  as  they  have  alternative  uses.  In 
fact,  there  are  a  large  number  of  alternative  uses  as  evidenced  by  the 
expansion  of  special  education  programs,  rental  of  rooms  to  other  public 
agencies,  or  the  closing  of  schools  and  their  rental  or  sale. 

6.  The  useful  life  of  school  facilities  is  taken  from  estimates  by  LAUSD 
administrators.  ^ 

7.  CCC  staff  gave  us  a  figure  of  6-10  years  depending  on  level  of  utilization 
and  assessments  of  technical  obsolenscence.    We  have  used  the  six  year 
figure  because  of  the  very  intense  level  of  utilization  of  the  equipment. 
However,  extending  the  estimated  life  to  ten  years  would  have  the  effect 
of  reducing  the  overall  instructional  costs  by  no  more  than  2-3  percent. 

8.  See  Levin  1975  and  Jamison  ejt  ajL. ,  1978. 

9.  Jamison  et.  al. ,  1970  suggests  that  at  that  time  a  cost  of  $50  per  session 
was  attainable  on  an  earlier  CCC  system.     That  estimate  seems  overly 
optimistic;  even  when  adjusted  for  inflation  it       about  half  of  out 
estimates.    Most  of  the  difference  appears  to  ari  e  from  the  fact  that 
coordinators  were  not  used  in  the  configuration  that  they  describe  as 
well  as  the  assumption  that  the  utilization  rate  would  be  25  sessions 
daily.     They  do  not  mention  the  number  of  minutes  per  session.  Early 
"drill  and  practice"  curricula  utilized  seven  minute  sessions,  and  they 
may  be  assuming  these  shorter  sessions. 
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TOWARDS  A  META  COST-EFFECTIVENESS  ANALYSIS  OF  EDUCATIONAL  INTERVENTIONS  .  ' 

I.  INTRODUCTION 

This  report  has  its  origin  in  earlier  work  that  was  done  on  the  cost 
analysis  of  a  particular  educational  intervention  using  computer-assisted 
instruction  (CAI)  to  improve  the  educational  performance  of  youngsters  who 
were  defined  as  educational  disadvantaged.     (Levin  and  Woo  1981)    Under  the 
sponsorship  of  the  National  Institute  of  Education  (NIE),  the  Los  Angeles 
Unified  School  District  (LAUSD)  ^nd  the  Educational  Testing  Service  (ETS) 
collaborated  in  a  unique  experiment  to  test  the  effectiveness  of  a  specific 
application  of  CAI  in  raising  the  test  scores  of  students  in  mathematics, 
reading  and  language  arts.    The  experiment  was  unique  in  that  it  was 
designed  to  examine  the  effectiveness  of  CAI  along  many  dimensions  including 
differences  in  amount  of  daily  instruction,  in  number  of  years  of  instruction, 
and  in  subject  area.     The  fact  that  the  experiment  was  carried  out  over  a 
four  year  period,  from  1976-1980,  meant  that  the  cumulative  effects    of  CAI 
could  also  be  examined. 

One  of  the  concerns  of  the  sponsoring  agencies  and  educational  policy-makers 
more  generally  is  the  question:    Is  CAI  more  effective  in  raising  academic 
performance  of  disadvantaged  youngsters,  relative  to  its. cost,  than  other 
educational  interventions?    Put  more  succinctly,  is  CM  a  more  cost-effectlveT 

/ 

strategy  than  other  educational  alternatives  for  impfroving  the  educational 
proficiencies  of  the  disadvantaged?    Since  the  purpose  of  the  ETS/LAtJSD 
experiment  was  to  examine  the  effectiveness  of  C^'I^^^^ha^t- par tlcular  information 
was  likely  to  derive  from  the  evaluation. 

In  order  to  provide  the  pertinent  data  on  costs,  a  special  study  was 
initiated  to  ascertain  the  cost  of  replicating  the  specific  CAI  approach  that 
was  taken  in  the  experiment.     Using  the  ingredients  or  resource  method  of 
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estimating  costs,  it  was  found  that  a  daily  ten  minute  session  could  be 

■    -  > 

provided  over  the  school  year  for  about  $135  per  student  in  additional  costs 
in  1978.     Given  the  funding  available  in  1978  under  Title  I  of  the  Elementary 
and  Secondary  Education  Act  of  1965,  about  $400  per  student,  as  many  as  three 
sessions  could  be  provided  per  child  on  different  subjects  or  a  single  subject. 
Or,  more  realistically,  two  sessions  coiild  be  provided  while  still  leaving 
about  $130  available  for  bther  Title  I  services. 

In  summary,  both  estimates  of  costs  and  effects  were  derived  for  the 
experiment.    But,  clearly  these  data  are  not  adequate  for  a  comparative  study 
of  cost-effectiveness  of  different  educational  interventions  without  similar 
information  on  other  educational  alternatives.    That  is,  knowing  both  the  cost 
and  effectiveness  of  different  amounts  of  CAI  in  different- subjects  is  not 
adequate  in  itself  forjudging  the  relative  cost-effectiveness  of  CAI.  Rather, 
we  also  need  similar  types  of  data  for  the  alternatives  to  CAI.    But,  given 
the  inability  to  Initiate  costly,  multi-year  experiments  for  the  educational 
in1ferv@ntl6ns,  such  data  cannot  be  derived  in  the  same  careful  and  painstaking 
way  that  they  were  derived  for  the  CAI. 

The  purpose  of  this  study  is  to  develop  an  approach  for  comparing  the 
cost-effectiveness  of  CAI  with  other  approaches  in  the  absence  of  such 
experimental  data.    The  appr^oach  builds  on  the  meta-analysis  strategies 
developed  originally  b-/  Gene  Glass  and  Richard  Light  and  their  colleagues 
and  refined  by-Gla'i^s,  McGaw  and  Smith  (1981).    The  idea  behind  meta-analysis 
is  that  we  can  learn  more  about  any  particular  phenomenon  by  drawing  upon  all 
of  the  evaluations  of  that  phenomenon  than  by  relying  on  any  single  one. 
Each  evaluation  contains  potential  information  that  others  may  not  have,  since 
each  is  based  upon  scope  and  treatment  conditions  that  are  likely  to  differ 
from  other  studie^.     For  example,  different  studies  will  analyze  different 
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intensities  and  versions  of  a  single  treatment  and^ill  apply  them  to 
different  populations.    A  generalization  about  the  phenomenon  should  draw 
upon  this  wide  and  richer  range  of  conditions  than  any  single  study  could,  _ 
encompass.  ^ 

The  evaluation  approach  that  tries  to  generalize. about  a  phenomenon 
from  judicious'analysis  of  all  of  the  studies  on  that  phenomenon  is  known 
as  a  m^ta-analysis.    This  does  not  mean  that"  all  studies  are  weighted  equally 
in  the  analysis.    Some  may  be  discarded  because  they  do  not  meet  even  minimal 
criteria  for  acceptability  on  the  basis  of  sucTT^factors  as- poor  design, 
inadequate  measurement,  and  so  on.    Of -course,  these  issues  often  are  matters 
of  degree.    That  is,  there  is  no  "perfect"  evaluation,  given  the  complexity 
of  the  world  in  which  social  phenomena  take  place.    But,  some  evaluations  are 
better  or  more  appropriate  than  others,  given  the  particular  question  of 
interest.    Meta-analysis  represents  an  attempt  to  assemble  a  large  range  of 
appropriate  studies  and  draw  generalizations  by  a  judicious  assessment  of 
them  to  see  what  type  of  pattern  seems  to  be  evident. 

Although  the  focus  of  this  study  is  on  the  development  of  a  meta-analysis 
for  cost-effectiveness  evaluation,  the  meta-analysis  of  effectiveness,  itself, 
will  not  be  a  focus.    Under  separate  arrangements.  Gene  Glass  of  the  University 
of  Colorado  has  been  working  on  a  parallel  study  of  meta-evaluations  of  the 
effectiveness  of  CAI,  reducing  class  size,  teacher  retraining  and  tutoring 
interventions.    All  of  these  represent  potential  alternatives  for  attempting 
to  improve  the  education  of  disadvantaged  youngsters.    This  particular  study 
will  focus,  instead,  on  developing  meta-evaluations  of  the  costs  of  these 
interventions  and .combining  them  with  the  Glass  estimates  of  effectiveness 
into  cost-effectiveness  measures  that  can  be  used  to  assess  the  available 
alternatives  that  might  be  considered  by  decision-makers. 
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This  report  will  proceed  in  the  following  way.    First,  we  will  discuss 
the  methodology  for  estimating  costs  and  its  applicability  to  a  meta-analysis 

c 

approach.     Specifically,  we  will  be  concerned  wi th  the  problems  that  arise  in 
this  type  of  application.     Second,  we  will  review  cost  models  for  the  four 
interventions  that  are  being  examined  by  Glass:     (1)    CAI;     (2)    reduction  in 
class  size;     (3)    teacher  retraining;    and  (4)     tutoring.     In  doing  this, 
we  will  focus  on  the  concepts  as  they  apply  to  each  type  of  intervention 
rather  than  the  final  cost  estimates.    Finally,  we  will  discuss  those  issues 
that  arise  in  bringing  these  together  in  a  policy  framework  with  the 
effectiveness  estimates  to  create  a  meta-analysis  of  cost-effectiveness. 
The  reader  of  this  report  should  be  warned  that  this  document  will  be  devoted 
primarily  to  conceptual  issues,  although  some  details  on  the  cost  models  will 
be  provided.    However,  the  actual  estimates  of  costs  and  cost-effectiveness 
will  be  the  subject  of  a  future  paper  rather  than  of  this  report. 
II.    DISCUSSION  OF  COST  METHODOLOGY 

The  appropriate  cost  methodology  for  addressing  this  type  of  problem  is 
known  as  the  ingredients  or  resource  approach.     (Levin  1976;  Levin  1981) 
This  approach  is  based  upon  two  essential  steps.    First,  an  inventory  is  made 
of  the  ingredients  or  resources  that  are  required  to  replicate  any  particular 
alternative,  for  example,  an  educational  intervention.    Second,  the  resources 
or  ingredients  that  are  required  are  given  a  cost  which  is  ba^ed  upon  the 
sacrifice  or  cost  to  society  of  using  the  resources  for  the  intervention  rather 
than  for  the  best  alternative  use.    A  third  step  that  is  sometimes 'taken  is 
to  distribute  the  costs  among  those  entities  bearing  them  such  as  different 
l^^vels  of  government,  volunteers,  and  other  private  sources.    Although  these 
steps  are  straightforward  in  principle,  they  are  rarely  straightforward  in 
practice  for  a  variety  of  reasons.    The  most  important  of  these  are  that 
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ingredients  for  replicating  an  intervention  are  not  always  obvious  or  provided 
in  any  systematic  way  for  the  analyst,  and  costs  are  often  found  to  be 
problematic  or  elusive.    Worst  of  all,  standard  accounting  approaches  and 
budgeting  techniques  used  in  the  educational  sector  will  not  provide  an 
accurate  picture  of  either  ingredients  or  costs.    Accordingly,  a  discussion 
of  the  principles  is  in  ordei^efore  proceeding  to  their  application  to  the 
present  case.  / 

The  basic  notion  behind  the  ingredients  method  of  costing  is  that  all 

/  ■       ■  ^-   ■  ■ 

resources  required  for  ^eplicating  an  intervention  must  be  accounted  for, 
and  the  proper  value  for  using  those  resources  for  that  intervention  rather 
than  an  alternative  use  must  be  established.     In  this  way,  it  is  possible  to 
establish  a  cost  value  that  represents  the  "sacrifice"  in  terms  of  value  of 

resources  that  society  gives  up  to  make  the  intervention.    Such  an  estimatie 

/ 

provides  a  consistent  picture  among  alternatives  on  the  nature  and  value  /of 
costs  as  well  as  an  appropriate  conceptual  framework  with  regard  to  the 
economics  of  costs. 
Identifying  Ingredients 

The  process  of  identifying  the  ingredients  for  any  particular  alternative 
begins  by  asking  the  question,  what  resources  are  required  to  replicate  a 
speci/ic  intervention  that  produced  the  effect  that  will  be  estimated.    That  is 
the  effectiveness  of  any  particular  intervention  is  based  upon  a  number  of 

ingredients  that  are  brought  together  and  organized  in  such  a  way  to  produce 

/ 

tl;<at  effect.    These  ingredients  include  personnel,  facilities ,  and  materials. 
Since  education  tends  to  be  labor-intensive,  a  substantial  aspect  will  be 
personnel  costs.    Here  we  are  concerned  not  only  with  the  amounts  of  personnel, 
but  also  their  qualities  or  characteristics. 

Some  of  the  ingredients  are  always  obvious,  and  some  are  not.    For  example 
the  principal  personnel  who  are  involved  in  the  intervention  and  the  materials 
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and  equipment  that  are  used  are  usually  readily  evident.  °  Less  obvious  are 
administrative  resources  and  facilities,  when  such  provisions  are  shared  with 
other  programs  or  functions.    However,  there  are  various  ways  of  estimating 
the  portion  of  shared  resources  that  should  be  linked  to  any  particular 
activity.    Finally,  there  are^l^dden"  ingredients,  those  that  are  not  apparent 
For  example,  in  experimen^^  interventions  the  experimenter  and  his  or  her 
staff  are  often  expect^  to  observe  the^experiment.     But,  what  if  the  presence 
of  the  experimentje^r  and  staff  serve  to  make  all  of  the  other  p^irsonnel  more 
highly  motivated  and  attentive^^an  they  would  be  in  '  the  absence  of  such 
distinguished  observers.     In  such  a  case,  the  personnel  associated  with  the 
research  function  are  indeed  required  to  replicate  the  intervention  as  it  is 
being  considered,  and  its  measured  effectiveness  is  due,  in  part,  to  these 
"hidden"  ingredients.  / 

9  /' 

Using  our  definition  of  ingredients  as  all  of  those  resources  that  produce 

■  / 

the  observed  effect  or  result,  all  of  these  ingredients  must  be  considered  in 
setting  out  the  overall  resource  requirements.    Replicating  the  intervention 
without  these  "hidden"  ingredients  is  likely  to  produce  a  different  effect. 
Even  if  we  consider  that  the  effect  of  having  experimenters  around  is  unintended 
or  a  hawthorne  effect  and  would  not  be  replicated  in  other  settings,  the 
specific  effects  that  we  will  be  measuring  will  be  based  upon  all  of  the 
ingredients  including  the  influence  of  the  experimenter^.    In  fact,  we  will 
suggest  below  that  the  teacher  retraining  interventions  seem  to  be  highly 
susceptible  to  the  influence  of  having  nationally  renowned  researchers  as 
observers. 

Finally,  even  ingredients  that  are  used  in  the  intervention  but  provided 
for  through  contributions  or  voluntary  personnel  are  identified.    The  fact 
that  any  particular  ingredient  is  paid  for  by  one  constituency  rather  than 
another  is  not  a  reason  for  omitting  it  from  the  specification  of  ingredients. 
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The  issue  of  who  pays  for  what  portions  of  an  intervention  is  an  important  one 
that  requires  subsequent /analysis,  but  no  ingredient  should  be  omitted  from 
the  analysis  because  it  is  "free"  to  the  sponsor  of  the  intervention.  All 
ingredients  or  resources  represent  a  cost  to  someone.    At  first  we  must 
concern  ourselves  only  with  identifying  the  ingredients  and  their  costs,  and 
later  we  can  examine  who  might  pay  for  them. 
^Costing  the  Ingredients 

After  identifying  the  ingredients,  it  is  necessary  to  determfne  their 
costs.    Although  an  accurate  cost  for  many  ingredients  will  be  found  in 
budget  statements,  many  costs  cannot  be  ascertained  in  that  way.     If  it  can 
be  assumed  that  personnel  are  receiving  the  salaries  and  other  benefits  that 
they  would  obtain  in  ^he  general  market  for  their  services,  such  information 
can  be  used  to  place  a  cost  on  these  inputs.    In  some  cases,  costs  will  not 
be  found  in  the  budgets,  and  in  other  cases  the  costs  will  be  incorrect. 
For  example;  volunteers,  contributed  facilities,  or  facilities  tbat  have  been 
paid  for  previously  will  not  be  evident  on  any  budget.    Yet,  because  such 
ingredients  have  alternative  uses  that  have  value,  there  is  a  cost  to  using 
them  for  the  intervention.    That- cost  can  be  ascertained  by  determinir.g  what 
similar  types  of  ingredients  would  cost  if  they  had  to  be  purchased. 

Budgetary  distortions  occur  when  expenditures  are  based  upon  special 
arrangements  of  a  non-market  nature  or  when  the  accounting  principles  utilized 
do  not  reflect  the  actual  use  pattern  of  the  resource.     In  the  first  case, 
one  government  such  a^  a  municipality  may  make  a  special  and  favorable  arrange 
ment  with  another  government  such  as  a  school  district  to  provide  a  surplus 
facility  at  a  bargain  price.    The  true  cost  or  value  is  the  amount  that  that 
facility  would  fetch  in  a  lease  or  rental  in  the  general  market.    In  this 
situation,  the  bargain  rate  would  understate  the  true  cost  or  value  of  the 
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facility..   The  most  common  distortion  in  budgetary  cofts  relative  to  true 
costs  is  reflected  in  the  situation  where  capital  improvements  are  charged 
to  the  budget  in  a  single  year  even  though  they  have  a  life  of  many  years. 
School  districts  and  other  units  of  government  typically  pay  for  equipment 
and  the  refurbishing  of  facilities  in  the  year,  that  such  improvements  were 
acquired.    Yet,  equipment  and  improvement  of  facilities  have  a  life  of  many 
years,  and  the  appropriate  cost  ^n  any  one  year  is  to  charge  for  only  their 
depreciation  and  a  rate  of  interest  on  the  remaining  investment. 

The  principles  of  estimating  costs  are  partially  based  upon  using  market 
values  and  partially  based  on  using  "shadow"  prices,  the  appropriate  value 
for  a  resource  if  it  were  traded  in  a  perfect  market.    In  order  to  ascertain 
how  these  principles  are  applied,  it  i§  necessary  to  know  both  economic  and 
accounting  concepts.    The  economic  framework'  provides  a  basis  for  ascertaining 
how  to  determine  the  cost  of  an  ingredient,  while  the  .accounting  framework 
sets  out  an  operational  principle  for  making  the  cost  estimate.  Economics 
without  knowledge  of  cost  accounting  provides  too  abstract  an  approach  for 
obtaining  concrete  cost  estimates;  and  accounting  without  the  conceptual 
framework  provided  by  economics  provides  too  applied  a  technique  with  all  of 
the  dangers  of  missing  ingredients  that  are  not  found  in  budgetary  or 
accounting  documents. 

Once  the  ingredients  are  identified  and  their  costs  determined,  it  is 
possible  to  determine  the  cost  of  each  intervention.    For  each  intervention 
a  list  of  ingredients  and  jippropriate  costs  is  compiled.     The  total  costs  of 
each  intervention  can  be  divided  by  the  number  of  students  served  or  the 
number  of  service  units  provided  to  give  a  cost  per  unit.     Or  alternatively 
a  cost  per  student  can  be  compared  with  a  measure  of  educational  effectiveness 
for  a  specified  population  for  /all  interventions  to  obtain  cost-effectiveness 
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measures  among  alternatives* 

One  additional  step. that  is  often  taken  is  to  ascertain  the  distribution 
of  the  cost  burden  among  interventions.     Consider  that  for  some  interventions 
the  school  district  must  pay  all  of  the  costs,  while  for  other  interventions 
it  is  possible  to  obtain  contributed  facilities  and  volunteers.     In  the  latter 
case,  some  of  the  costs  are  borne  by  those  who  are  volunteering  or  providing 
the  facilities.     In  that  case,  the  cost  might  be  less  to  the  school  district 
for  the  latter  intervention,  even  if  its  total  costs  to  all  of  the  payors  is 
greater  than  in  the  first  intervention.    Obviously,  the  cost  to  the  decision- 
maker will  weigh  heavily  in  his  or  her  choice,  regardless  of  the  total  costs 
of  the  intervention.    Accordingly,  a  distributional  analysis  of  cost  burden 
is  carried  out  to  determine  the  costs  to  different,  constituencies  of  each 
intervention. 
Meta-Analysis  of  Costs 

A  meta-analysis  of  costs  begins  with  two  major  challenges.     The  first  is 
characterized  by  the  normal  hazards  of  meta-analysis.     Such  hazards  include 
the  attempt  to  combine  the  results  of  a  large  and  diverse  set  of  studies 
carried  out  on  different  populations  with  different  designs  and  objectives. 
But,  in  addition  to  these  challenges,  there  is  the  additional  one  reflected 
by  the  fact  that  virtually  all  evaluations  of  educ^tional  interventions  lack 
any  cost  perspective.     In  the  meta-evaluations  of  educational  interventions, 
at  least,  all  of  the  studies  will  focus  on  a  relatively  common  criterion  of 
outcome.     But,  there  is  no  comparable  concern  for  estimating  costs.    The  result 
is  that  such  information  is  lacking  in  its  most  rudimentary  f.orm. 

To  begin  with,  most  such  studies  include  no  discussion  of  costs  or  cost 
factors.     Even  worse,  few  have  any  systematic  and  reasonably  complete 
description  of  the  intervention  and  the  ingredients  that  are  required.  This 
tends  to  be  a  deficiency  of  many  evaluations  in  education.    That  is,  the 


-10- 

trea^mentr  is  described  in 'an  idealized  form,  with  little  attention  to  the 
acpifal  treatment  that  was  received  by  students  and  with  no  serious  attempt 
to  describe  carefully  the  ingredients  that  were  used.    The  most  severe 
shortcoming  of  evaluations  of  educational  interventions  from  the  perspective 
of  doing  cost  analysis  is  this  dearth  of  relevant  information.    That  is, 
somehow  through  the  scrutiny  of  a  number  of  similar  types  of  intervention 
studies  and  background  reading  on  the  intervention,  the  cost  analyst  must 
construct  a  reasonable  cost  model  consisting  of  the  ingredients  required  for 
the  intervention  and  the  probable  costs  of  the  ingredients.    To  a  large  degree 
this  exercise  can  be  only  minimally  informed  by  any  specific  study. 

A  related  problem  is  the  one  that  was  mentioned  above.    Hidden  ingredients 
represent  a  real  possibility  in  a  situation  in  which  there  is  no  systematic 
attempt  to  provide  information  on  the  treatment  and  ingredients  as  a  central 
part  of  each  study  being  reviewed  in  the  meta-analysis.    Not  only  is  the 
cost  reconstruction  susceptible  to  omitting  hidden  .ingredients,  but  it  must 
also  risk  guessing  which  ingredients  are  necessary  for  replication.     In  the 
teacher  retraining  studies,  observers  from  projects  that  were  directed  by 
nationally-known  researchers  in  teacher  effectiveness  were  ixsed  to  determine 
if  the  teachers  were  utilizing  their  training.     In  the  evaluations  of  these 
interventions,  such  observers  are  treated  as  part  of  the  data-gathering 
apparatus  rather  than  as  an  intrinsic  input  into  the  intervention.    Yet,  it 
would  seem  unreasonable  to  assume  that  the  effects  that  are  observed  could 
be  replicated  without  observers  deriving  from  major  research  organizations. 

In  the  longer  run,  researchers  should  be  expected  to  provide  explicit 
detail  on  the  nature  of  interventions,  with  special  attention  devoted  to  ' 
specific  ingredients.    Further,  they  should  be  required  to  separate  both 
conceptually  and  empirically  the  ingredients  that  are  developmental  and 
experimental  on  the  one  hand  from  those  that  would  be  required  for  replication. 
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Many  research  studies  that  attempt  to  ascertain  the  effectiveness  of  Inter- 
ventions utilize  both  kinds  of  Inputs.    For  example,  the  construction  of 
program  materials,  development  of  program,  and  training  of  observers  and 
researchers  are  not  required  for  replicating  an  Intervention.    Yet,  often 
portions  of  these  activities  are  found  to  be  enextrlcable  In  the  presentation 
of  research  from  those  factors  that  would  be  required  for  replication. 
A  clear  distinction  ought  to  be  drawn  between  the  two  In  writing  research 
reports.     Even  more  basic,  all  research  reports  should  Include  a* systematic 

presentation /of  the  types  and  amounts  of  resources  that  were  used  In  the 
^  / 

lnterventlon\ 
III.     SPECIFIC  INTERVENTIONS 

The  purpose  of  this  section  of  the  report  Is  to  review  fouif  specific 
Interventions  with  respect  to  the  elements  of  both  their  Ingredients  and 
cost  structures.    This  review  Is  based  upon  a  review  of  studies  that 
attempted  to  evaluate  the  Intervention.     In  each  case  we  will  describe  briefly 
the  Intervention  and  discuss  Its  Ingredients.    We  will  devote  special  attention 
to  the  variants  of  the  cost  model  with  respect  to  each  Intervention. 

1 .     Computer-Assisted  Instruction 

The  plethora  of  computer-assisted  Instructional  approaches  makes 
generalization  extremely  difficult.    They  vary  from  specific  Instructional 
units  that  can  be  used  to  supplement  regular  Instruction;  to  complete 
courses  that  are  taken  by  computer;  to  continuous  and  year-long  sequences 
providing  drill  and  practice  In  support  of  the  on-going  Instructional  program 
In  particular  subjects.     Each  of  these  approaches  has  different  objectives 
as  well  as  providing  a  plethora  of  alternatives  with  regard  to  hardware, 
curriculum,  and  organization.    As  such,  it  is  not  possible  to  refer  to  CAI 
as  an  Instructional  intervention  that  has  great  specificity.     In  many  respects, 
the  different  forms  and  uses  of  CAI  represent  different  instructional 
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interventions,  as  different  as  the  diversity  among  different  curricular 
Interventions.    Accordingly,  the  CAI  model  will  not  be  a  general  one,  but  '  " 
the  particular  one  that  was  used  as  the  basis  for  the  ETS/LAUSD  experiment- 

This  model  has  the  virtues  of  having  been  tested  for  a  numbed  of  ^ 
subjects  (mathematics,  reading  and  language  arts);  most  of  the  primary  ' 
school  grades;  different  intensities  of  treatment  as  well  as  durations  pf 
up  to  three  years;  and  an  excellent  experimental  and  analytic  design. 
Although  the  intervention  addresses  only  the  drill  and  practice  capability 
of  CAI,  this  is  an  important  application  and  one  that  competes  with  other 
types  of  educational  interventions  in  terms  of  attempting  to  improve  basic 
.skills.    The  resource  cost  model  Is  approximated  in  Table  One.  ' 

Table  One 

Ingredients  of  Resource  Cost  Model  for  CAI 

Facilities  -  ^Classroom  containing  terminals  and 

CAI  equipment  as  well  as  renovations 
necessary,  such  as  air-conditioning 
and  security  devices      \;  ~ 

/  ^     Equipment  -  Central  processing. unit,  terminals,  , 

printer,  ari,d  toramunications  equip- 
ment-if  needed,  such, as  telephone 
lines,  modem's. and  multiplexers 

Curriculum  -  Rental  of  software  required  for  • 

instruction 

Personnel  -  Coordinators  and  aides  as  welL  as 

training 

Miscellaneous  -      Administrative,  insurance, 
utilities,  etc. ... 

•  to  ■ 

Table  One  shows  the  basic  elements  of  the  CAI  model  used  for  this  analysi 
Most  of  the  ingredients  are  self-explanatory.     The  central  processing  unit 
and  student  terminals  are  situated  in  a  special  classroom  that  is  air- 
conditioned'  and  has  special  security  arrangements  to  protect  the  equipment. 
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The  equipment  consists  of  the  central  processing  unit  (CPU)  that  stores  the 
curriculum  and  student  records* and  that  provides  the  instruction  and  student 
terminals  with  keyboards  and  video  screens.    If  the  CPU  is  shared  >y  more 
than  one  school,  there  are  also  costs  for  the  communications  equipment 
between  schools.    The  curriculum  includes  the  computer  programs  that  are 

leased  for  use  in  the  system.    Personnel  include^^the  coordinator  responsible 

X 

for  each  CAI  room  and  aides  who  answer  student  questions  and  assist  the 

\  ■ 

coordinator.    Finally,  there  are  a  number  of  miscellaneous  ingredients  such 


as  the  administrative  inputs,  insurance  and  utilities 


Since  the  costs  of  this  CAI  approach  have  already  ^een  described  and 


estimated  in  detail,  it  is  best  to  refer  to  the  more  compiiehensive  study. 
In  19/8,  these  costs  were  estimated  at  about  $135  per  student  on  an  annual 
basis  for  each  ten  minute  daily  exposure  to  CAI.     In  later  comparisons  of 
costs,  it  is  these  estimates  that  will  be  used  to  compar^^with  those  of  other 
interventions.  > 
2.     Reduction  in  Class  Size 

/ 

Probably  the  most  universal  and  common  strategy/ for  improving  student 
academic  performance  is  that  of  reducing  class  si^^.     Presumably  the 
reduction  of  class  size  provides  a  number  of  b^efits  to  students.  First, 
students  may  feel  that  instruction  is  mor^xpersonalized  and  less  anonymous 
Ati  smaller  classes.    Thus,  student^-'My  feel  more  comfortable  in  the 
classroom  environment,  and^te^chers  can  tailor  instruction  more  to  the 
specific  students  in  the  class.    A  second  and  related  possible  benefit  is 
that  smaller  classes  do  not  require  as  much  regimentation  to  keep  order. 
To  the  degree  that  less  time  can  be  taken  on  the  establishment  and  reinforce- 
ment  of  rules  for  maintaining  order,  there  is  more  time  for  instruction. 
Finally,  the  smaller  the  class, ^  the  more  time  that  teachers  can  devote  to 
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individualized  instruction  for  each  child  during  those  parts  of  the  teaching 
process  that  enable  individual  attention.        ^  ' 

While  smaller  class  sizes  have  this  potential  for  improving  instruction 
and  academic  achievement,  it  is  not  clear  that  they  always  realize  this 
potential.    Moreover,  evaluations  of  their  effects  vary  in  rigor  and  the 
control  of  extraneous  influences.    Recently,  Gene  Glass  and  his  associates 
have  carried  out  a  meta-analysis  of  the  effects  of  class  size  on  achievement. 
This  analysis  suggests  that  reductions  in  class  size^do  have  modest  effects 
on  achievement,  with  the  size  of  the  effect  dependent  on  the  magnitude  of 
the  reduction  and  the  initial  class  size  that  is  being  reduced. 

The  ingredients  model  for  reducing  class  size  is  relatively  straight- 
forward, encompassing  two  resources,  teachers  and  facilities.  Obviously, 
as  a  given  student  population  is  divided  into  more  and  more  classroom  units, 
more  teachers  are  needed  and  more  classrooms  are  needed.     Thus,  the  resource 
cost  model  for  any  reduction  in  class  size  must  estimate  its  impact  on  these 
two  ingredients.     By  costing  out  the  two  ingredients,  the  cost  of  any  reducti( 
in  class  size  can  be  estimated. 

3.    Tutoring  . 
in  a  sense, 'tutoring  represents  the  extreme  reduction  in  class  size  by 
reducing  the  number  of  pupils  for  each  instructor  to  a  single  person  or  just 
a  few.    However,  mor^  typically  tutoring  interventions  do  not  fit  well  into 
the  reduction  of  class  size  paradigm  because  they  are  devoted  to  remedial' 
instruction  or  to  subjects  not  covered  in  the  normal  curriculum  such  as 
advanced  study  for  the  gifted.    Or,  they  can  be  used  to  simply  reinforce  what 
is  covered  in  the  regular  course  of  instruction.    Moreover,  they  are  rarely 
taught  by  classrodm  teacliers.     Rather,  the  bulk  of  tutoring  interventions 
in  the  public  schools  are  based  upon  students  of  the  same  age  tutoring  others 
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(peer  tutoring)  or  older  students  tutoring  younger  ones  (cross  age  tutoring). 
The  usuinr  arrangement  is  to  choose  a  student  with  greater  proficiency  to 
assist  one  whose  achievement  has  been  lover  in  the  peer  tutoring  approach. 
In  some  cases,  parents  or  other  adults  have  been  used  to  totor  youngsters. 

The  main  advantage  of  the  student  tutoring  model  is  that  students 
represent  a  large  potential  teaching  force.    Thus,  they  can  be  an  imports^ 
resource  in  the  instructional  process.    Second,  studies  have  suggested  that 
the  tutors  may  benefit  substantially  in  both  self-esteem  and  in  raising 
their  own  achievement  levels  by  tutoring  others,  so  both  the  tutor  and  the 
tutee  may  benefit  from  this  process.    Third,  the  approach  has  a  high  degrer^ 
of  flexibility  in  terms  of  subjects,  tutors,  pupils,  required  facilities, 
and  so  on.    The  approach  can  be  highly  formal  with  substantial  training  of 
tutors,  structured  exercises,  tailored  materials,  and  special  facilities  or 
highly  informal  with  little  or  rto  training  or  structure  and  use  of  regular 
classroom  materials  as  well  as  reliance  on  available  space  in  hallways  or  ^ 
other  parts  of  the  building.    Finally,  tutoring  can  be  occasional  for  students 
who  have  difficulties  with  particular  concepts  or  exercises,  or  it  can  be 
systematic  for  a  given  subject  or  for  all  subjects.    Thus,  even  the  amount  of 
tutoring  received  by  participating  students  can  differ  from  a  few  hours  to 
more  than  100  hours  a  year. 

The  flexibility  and  richness  of  the  tutoring  model  make  it  an  attractive 
potential 'intervention  for  schools,  but  the  variety  of  applications  creates 
obstacles  to  the  construction  of  a  straightforward  resource  cost  model  to 
♦   represent  t.he  approach.    This  can  be  readily  discerned  from  Table  Two  which 
shows  the  ingredients  for  t^ie  tutoring  model.    Personnel  include  the 
supervisor  or  coordinator  and  tutors.     But,  the  amount  and  qualities  of 

s 

personnel  depen^d  upon  the  specific  tutoring  approach.     For^  example,  the  use 
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of  adults  versus  older  students  versus  peers  will  have  profound  effects  on 
.  the  estimation  of  costs.    From  the  perspective  of  students,  the  value  of 
student  time  will  depend  on  whether  the  activity  takes  place  during  school 
hours  or  after  school.    Presumably,  the  "time  of  a  student  tutor  should  be 
evaluated  only  on  the  basis  of  the  amount  of  learning  that  he  or  she  foregoes 
during  the  school  day.    But,  after  school  hours  there  is  a  value  that  is 
determined  by  employment  possibilities  or  the  value  of  other  voluntary 
activities. 

Table  Two 

Ingredients  of  Resource  Cost  Model  for  Tutoring 

Personnel  -  -  Supervisor  or  coordinator 

-  Tutors 

Training  -  -  Time  of  trainer  and  tutor  used 

for  training  process 

-  Materials  used  for  training  and 
other  ingredients 

Facilities  -  Tutoring  space 

Materials  -  Specialized  materials  used  for 

tutoring  (beyond  those  used 
for  regular  instruction) 

Other  factors  determining  the  personnel  input  include  the  length  of 
'tutoring  sessions  and  whether  the  tutoring  is  supplementary  or  used  for 
replacing  regular  instruction.    The  amount  of  supervision  or  coordination 
is  also  affected  by  these  factors  as  well  as  the  amount  of  training  that 
musf^be  given.    Training  differs  substantially  among  different  tutoring 
approaches,  with  obvious  consequences  for  costs.     The  implications  of 
facilities  and  materials  on  costs  is  also  one  subject  to  great  variability. 
For  example,  among  23  tutoring  studies,  the  range  in  tutoring  time  over  a 
year  varied  from  one  hour  to  180  hours  with  the  median  at  only  2  hours  a  year. 


Training  time  varied  from  none  to  37  hours  a  year,  with  a  median  of  less 
than  one  half  hour.  Similar  variation  is  found  in  the  use  of  facilities 
and  materials. 

Accordingly,  cost  models  must  be  constructed  that  permit  different 
configurations  of  ingredients  to  match  the  evaluations  of  effectiveness. 
A  number  of  possibilities  exist  including  providing  estimates  of  costs  for 
different  variants  of  the  tutoring  model  as  well  as  specific  cost  estimates 
for  those  models  that  have  been  characterized  by  the  most  credible  evaluations 
or  other  criteria  such  as  implementability. 

4.    Teacher  Retraining 

A  substantial  part  of  the  educational  enterprise  has  been  devoted  to 
teacher  retraining.    The  assumptions  underlying  this  intervention  are  that 
teachers  have  been  poorly  prepared  in  the  past,  or  that  new  evidence  on 
teaching  effectiveness  has  been  discovered  that  should  be  conveyed  to 
teachers  to  alter  teaching  practice.    Without  commenting  on  this  rationale, 
it  seems  peculiar  that  teacher  retraining  should  always  take  as  given  the 
inadequacies  of  existing  teacher  training  without  altering  the  basic  training 
itself.     That  is,  if  teacher  retraining  represents  an  important  "repair"  to 
make  up  for  the  inadequacy  of  teacher  selection  or  teacher  education,  then 
it  would  appear  that  the  longer  run  solution  is  to  improve  teacher  education 
and  selection.    However,  many  schools  are  faced  with  the  dilemma  of  having 
large  numbers  of  tenured  teachers  who  could  be  far  more  effective  than  they 
presently  are.    The  challenge  is  to  find  a'  retraining  intervention  that  will 
improve  their  performance  in  raising  achievement  levels  of  their  students. 

l^ile  there  is  a  long  history  of  such  efforts,  the  oVerall  picture  of 
teacher  retraining  is  a  glum  one.     In  general,  theVe  has  been  little  evidence 
that  such  interventions  make  much  of  a  difference  in  teacher  or  student 
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:  behavior.    However,  the  last  few  years  have  witnessed  a  number  of  projects 
which  have  claimed  success  in  improving  the  capacity  of  teachers  to  raise 
the  achievement  of  pupils.     In  general,  these  approaches  have  in  common  the 
goal  of  getting  teachers  to  divide  the  curriculum  into  highly  structured 
unlts^ in  which  goals  are  made  clear;  general  principles  are  advanced; 
specific  exartples  are  given  followed  by  exercises  with  feedback  to  students; 
and  finally  an  assessment  of  student  performance  is  made.     The  teacher 
retraining  intervention  was  disarmingly  simple.    Teachers  were  given  a 
training  manual  that  provided  guidance  for  structuring  lessons  in  mathematics 
Training  sessions  were  given  to  introduce  teachers  to  the  manual  and  its 
concepts  and  applications.     Finally,  teachers  were  observed  to  see  if  they 
were  implementing  the  directions  set  out  in  the  manual. 

Table  Three 

Ingredients  of  Resource  Cost:  Model  for  Teacher  Retraining 

Personnel  -  -  Trainer 

-  Teachers  during  training 

-  Observers 

Materials  -  Manual  and  other  related 

materials  ^ 

Facilities  -  Space  for  training 

Table  Three  sets  out  the  rather  simple  list  of  ingredients  for  this 
retraining  approach.    Personnel  include  the  trainer,  the  t^e  of  the  teachers 
during  the  training  period,  and  the  classroom  observers.    Although  the 
research  studies  stipulated  that  the  observers  were  not^  a  part  of  the  inter- 
vention, but  were  only  used  to  systematically  gather  data  on  teacher  practices 
it  is  difficult  to  believe  that  the  observers  did  not  have  an  effect  on  the 
teachers  whom  they  were  observing.     Sponsors  of  the  projects  were  some  of  the 
leading  national  figures  in  teaching  effectiveness,  and  their  students  and 
colleagues  served  as  observers.    Accordingly,  it  is  hard  to  argue  that  they 
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were  unobtrusive  and  not  part  of  the  "treatment."    It  is  more  reasonable  to 
believe  that  teachers  wanted  to  impress  the  observers  with  their  preparation, 
teaching  prowess,  and  adherence  to  the  manuals.    Accordingly,  they  are 
included  as  an  input  into  this  model. 

Among  the  four  projects,  training  time  varied  from  three  hours  to  360 
hours  a  year,  with  a  mean  of  97  hours.    Observer  time  varied  from  3  hours  to  36 
hours  per  year.    The  manual  was  relatively  brief,  only  20-30  pages,  and  our 
main  concern  is  the'  replication  or  reproduction  costs  of  this  ingredient. 
Since  there  were  only  four  projects,  it  seems  sensible  to  estimate  the  costs 
of  each      -  that  is,  to  view  them  as  four  separate  models  to  compare  with 
the  four  separate  studies  of  effects. 
Costing  the  Ingredients 

At  the  present  time  attempts  are  being  made  to  ascertain  the  costs  of 
each  of  the  ingredients.    These  will  be  estimated  for  each  ingredient 
separately  and  aggregated  for  each  model  by  major  category  of  input  as 
reflected  in  the  earlier  work  of* Levin  and  Woo  (1981).    Although  the  costs 
of  specific  ingredients  may  differ  from  locality  to  locality  based  upon 
local  markets,  it  is  important  to  base  costs  on  some  "average"  figure. 
Of  course,  the  cdst  estimates  can  be  modified  to  take  account  of  differences 
on  a  focal  level,  but  a  meta-analysis  should  provide  an  overall  or  general 
picture.     la  the  final  section  of  this  report  we  will  consider  some 
implications  for  a  cost-effectiveness  analysis. 

IV  ■  COMBINING  COSTS  AND  EFFECTS  INTO  COST-EFFECTIVENESS  MEASURES 

The  final  step  is  that  of  combining  the  cost  estimates  generated  by 
these  models  with  the  effectiveness  estimates  generated  by  Gene  Glass  into 
ratios  of  cost-effectiveness  that  can  be  compared  among  interventions. 
For  example,  the  Levin-Woo  estimates  of  costs  for  CAI  for  1978  were  about  $135 
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per  ten-minute  session  on  an  annual  basis.    According  to  the  Final  Report 
of  the  ETS/LAUSD  project,  such  an  intervention  seemed  to  have  the  effect 
of  increasing  test  scores  in  arithmetic  computation  for  grades  1-6  by  .36 
standard  deviations  after  one  year  of  CAT,   .56  standard  deviations  after  two 
years  of  CAT,  and  .72  standard  deviations  after  three  years.     (p.  164) 
According  to  Gene  Glass,  one  standard  deviation  is  about  equivalent  to  one 
year  of  instruction  at  this  level.    Thus,  we  would  like  to  compare  a  gain 
of  ^36  standard  deviations  for  a  one  year  intervention  of  CAT  at  a  cost  of 
$135  with  the  cost  of  obtaining  similar  gains  through  other  interventions. 
Presumably  some  interventions  will  be  found  to  provide  larger  effects  per 
unit  of  ,cost  than  others.    Moreover,     if  the  differences  are  very  large,  we 
should  consider  seriously  that  the  differences  in  cost^ef f ec tiveness  are 
an  important  basis  for  selecting  some  interventions  over  others.  However, 
a  number  of  ^challenges  and  possibilities  will  be  present  that  need  discussion. 
Joint  Products  or  Multiple  Outputs 

One  of  the  major  challenges  to  the  analysis  will  be  that  most  of  the 
evaluations  are  based  on  the  examination  of  a  single  output.    Yet,  some 
interventions  may  produce  changes  in  student  achievement  and  other  student 
outcomes  along  mojre  than  one  dimension.    An  example  is  instruct/,vQ.  Suppose 
we  compare  two  interventions  on  the  basis  of  their  costs  and  effectiveness  in 
improving  mathematics  test  scores,  CAI  and  reduction  of  class  size.     The  CAT 
will  be  tailored  to  produ'ce  improvements  in  mathematics  proficiency,  since  the 
children  will  be  exposed  only  to  arithmetic  drill  and  practice.     In  contrast, 
the  reduction  of  class  size  will  presumably  affect  all  classroom  activities, 
both  the  teaching  of  mathematics  and  other  subjects.    That  is,  it  is  probably 
fair  to  assume  that  the  intervention  of  a  drill  and  practice  curriculum  in 


mathematics  for  CAI  should  be  evaluated  primarily  on  the  basis  of  its  costs 
^  and  effectiveness  with  respect  to  the  mathematics  outcome  alone.    One  can 
search  for  other  effects  or  joint  outputs  such  as  improvement  in  other 
subjects  or  in  self-concept  as>  a  result  of  the  CAI  sessions  in  mathematics. 
But,  if  no  other  effects  are  found,  it  is  probably  correct  to  assign  the 
entire  cost  of  the  CAI  sessions  in  mathematics  outcome  alone. 

But,  in  the  case  of  a  reduction  in  class  size,  clearly  only  a  portion 
of  the  cost  oif  the  intervention  should  be  assigned  to  the  cost  of  mathematics 
instruction.     Typically,  only  about  one  hour  out  of  a  five  or  six  hour  day 
in  the  elementary  school  curriculum  is  devoted  to  mathematics.  Accordingly, 
only  that  portion  of  costs  of  reducing  class  size  shotild  be  assessed  as  the 
cost  of  the  class  size  intervention  for  improving  mathematics  achievement. 
At  least  five/sixths  of  the  time  over  which  a  reduction  in  class  size  is 
operative  is  associated  with  other  activities.    The  estimation  of  the  cost- 
effectiveness  ratio  for  improving  mathematics  achievement  through  the 
reduction  of  claSs  size  must  take  account  only  of  the  portion  of  the  inter- 
vention attributable  to  the  teaching  of  mathematics.     Other  work,  such  as 
that  of  Dean  Jamison  on  cost-effectiveness  of  class  size  and  other  interventions, 
has  tended  to  overstate  the  appropriate  cost  for  reducing  class  size  by 
assigning  all  of  it  to  a  narrow  instructional  domain  that  comprises  a^ small 
part  of  the  curriculum. 

To  summarize,  each  intervention  will  hav6  to  be  evaluated  to  ascertain 
if  all  of  the  cost  should  be  assigned  to  the  specific  domain  in  which 
effectiveness  is  being  evaluated  or  only  a  portion  of  the  cost.    An  alternative 
procedure  is  to  specify  a  number  of  potential  outcomes  of  the  intervention 
including  changes  in  achievement  in  a  number  of  subject  areas.  These 
evaluations  can  then  take  account  of  multiple  outcomes  which  can  be 
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aggregated  l/nto  a  single  metric  of  effectiveness  by;  applying  utility 
weightings /to  the  outcome.    By  using  different  utility  weights  to  value  the 
outcomes,  the  results  can  be  subjected  to  sensitivity  analyses  and  tests  of 
robustnss^.    Each  utility  index  can  be  set  against'  the  total  cost  for 
producing  that  bundle  of  outcomes  in  the  cost-effectiveness  comparisons  as 
illustrated  in  Levin  1976  and  1981. 

Thi|s,  the  analysis  of  effectiveness  for  tutoring  can  include  the 
improveiients  in  achievement  for  both  the  tutors  and  their  studgnts,  relative 
to  what  their  achievement  levels  would  be  in  the  absence  of  the  tutoring 
activity.    Reduction  in  class  size  can  be  used  to  evaluate  changes  in 
achievement  for  the  major  subject  domains,  and  so  on.    The  basic  problem 
with  this  approach  is  that  not  all  of  the  evaluations  choose  to  assess  more 
than  one  area  of  outcome,  and  when  multiple  outcomes  are  assessed  among 
interventions,  they  do  not  necessarily  overlap  among  studies.     For  this 
reason,  the  apportioning  of  costs  according  to  the  use  of  resources  for 
producing  a  single  output  is  probably  a  more  attainabl^  procedure.    Of  course, 
even  this  approach  assumes  that  joint  costs  of  productidn  are  minimal, 
that  is,  that  the  costs  of  intervention  can  be  separated \according  to  the 
portion  of  them  that  are  linked  to  a  particular  outcome.    Wether  this  is 
realistic  is  problematic. 
Cost-Effectiveness  Within  Interventions 

A  final  possibility  that  is  being  explored  with  the  data  is  that  of 
cost-effectiveness  analysis  of  variants  of  each  model  that  permit  comparison 
both  among  interventions  and  within  them.    For  example,  the  large  number 
of  tutoring  studies  may  enable  some  analysis  of  how  both  costs  and 
effectiveness  vary  as  one  increases  the  amount  of  tutoring,  amount  of 
supervision,  training  time,  and  the  extensiveness  of  materials.    In  the 


standard  case  familiar  to  economics  we  have  two  equations: 

,  (1)    A  «  f     (X-^,  Xy  X^) 

(2)    B  -  p^X^  +  P2X2  +  P3X3  +  P4 

(1)  corresponds  to  the  standard  production  function  concept  in 
economics  where  the  achievement  of  a  student  is  a  function  of  X^  (hours  of 
tutoring  received),  X2  (amount  of  supervision),  X^  (training  time  for  tutors), 
and  X,   (extensiveness  of  materials).    It  is  assumed  that  A  is  an  increasing 
function  of  each  of  the  inputs,  but  that  the  law  of  diminishing  marginal 
returns  holds  for  each  input  so  that  at  some  point  the  additional  achievement 
for  a  unit  increase  in  each  input  (holding  the  others  constant)  begins  to 
diminish.     In  the  terminology  of  calculus,  the  first  partial  derivatives  are 
assumed  to  be  positive  and  the  second  are  assumed  to  be  negative. 

(2)  represents  the  budget  equation.     Given  any  level  of  budget  for  use 
on  an  intervention,  the  entire  budget  is  allocated  among  the  inputs  according 
to  the  expenditure  on  each  input  which  is  determined  by  its  price  (p)  and  the 
amount  of  the  input  (X)  that  is  utilized.    The  familiar  form  of  the  problem 
is  to  maximize  (1)  subject  to  the  constraints  of  (2).    The  standard  solution 
that  derives  from  the  lagrangian  approach  to  constrained  maximization  is  that 
each  of  the  inputs  will  be  utilized  until  that  point  where  the  additional 
contribution  to  A  (achievement)  for  the  last  unit  of  X  relative  to  the  price 
of  X  will  be  equal  for  all  inputs  as  in  (3). 
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Using  the  work  of  Glass  in  reviewing  the  evaluations  of  tutoring,  it 
might  be  possible  to  make  some  estimate  of  the  parameters  of  (1);  on  the 
basis  of  the  cost-analysis  we  can  estimate  the  parameters  of  (2).  These 
can  then  be  combined  to  make  estimates  of  variants  of  tutoring  with  respect 
to  costs  and  effects  of  different  combinations  of  the  inputs. 
SuTimiary 

The  purpose  of  this  report  was  to  suggest  how  a  meta-analysis  of  costs 
could  be  constructed  and  combined  with  a  meta-analysis  of  effectiveness  of 
educational  interventions  to  provide  a  meta-analysis  of  cost-effectiveness 
comparisons.     Clearly,  this  will  be  a  first  attempt  at  bringing  together 
a  wide  variety  of  information  from  diverse  studies  in  this  form.    As  such, 
it  should  be  considered  tentative  and  provocative  rather  than  definitive. 
Yet,  if  we  are  to  benefit  from  an  accumulation  of  knowledge  that  will  inform 
policy  decisions  within  a  framework  of  limited  resources,  this  is  clearly 
the  direction  that  we  must  go.    A  future  report  will  provide  the  first 

# 

estimates  using  the  techniques  set  out  in  this  paper. 
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